In both the split and the separate seesaw models, three generations of a Dirac spinor, $\Psi_I^5(x, y) = (\chi_I^5(x, y), \bar{\psi}_I^5(x, y))^T$, are introduced on the bulk with bulk masses $m_{5,I}$ in this flat five-dimensional space-time, where $I$ denotes the generation index as $I = 1, 2, 3$. A fundamental action for the Dirac spinors is

$$S = \int d^4 x \int_0^\ell dy M (i \overline{\Psi}_I^5 \Gamma^A \partial_A \Psi_I^5 + m_{5,I} \overline{\Psi}_I^5 \Psi_I^5), \quad A = 0, 1, 2, 3, 5, \quad (7.1)$$

where $M$ is a five-dimensional fundamental scale, the Dirac mass matrix $m_5$ is assumed to be diagonal ($m_{5,I} \equiv (m_5)_{II}$) for simplicity, and $\Gamma^A$ are the five-dimensional gamma matrices. The five-dimensional Dirac equations determine wave function profiles of zero modes on the bulk as $e^{\mp m_{5,I}y}$ for $\chi_I^5(x, y)$ and $\bar{\psi}_I^5(x, y)$. Note that only $\bar{\psi}_I^5(x, y)$ can have a zero mode under an orbifold parity as $\Psi_I^5(x, y) \to P\Psi_I^5(x, y) = +\Psi_I^5(x, y)$ with $P \equiv -i\Gamma_5$. After the canonical normalization of the Dirac spinor in four dimensions, the zero modes of $\Psi_{R_I}^5(x, y) \equiv (0, \overline{\psi}_I^5(x, y))^T$ are given by

$$\Psi_{R_I}^{(0)}(x, y) = \sqrt{\frac{2m_{5,I}}{(e^{2m_{5,I}\ell} - 1)M}} e^{m_{5,I}y} \psi_{R_I}^{(0)}(x), \quad (7.2)$$

where $\psi_{R_I}^{(0)}(x)$ are identified with the normalized right-handed neutrino fields in four dimensions, $\psi_{R_I}^{(0)} = \nu_{R,I}$. One can see that the five-dimensional wave function profiles of $\Psi_{R_I}^{(0)}(x, y)$ with positive (negative) value of $m_{5,I}$ localize at the hidden (SM) brane since the profile is proportional to $e^{m_{5,I}y}$. Thus, the right-handed neutrinos have exponentially suppressed Yukawa couplings at the SM brane when $m_{5,I}\ell \gg 1$ with positive $m_{5,I}$.

The relevant action for the split seesaw model, that uses the above property of the wave function profiles of the right-handed neutrinos, is given by

$$S = \int d^4 x \int_0^\ell dy \left[ M \left( i \overline{\Psi_{R_I}^{(0)}} \Gamma^A \partial_A \Psi_{R_I}^{(0)} + m_{5,I} \overline{\Psi_{R_I}^{(0)}} \Psi_{R_I}^{(0)} \right) \right. \\ \left. - \delta(y) \left( (\tilde{F}^\dagger)_{I\alpha} \overline{\Psi_{R_I}^{(0)}} l_{L\alpha} \tilde{\Phi}^\dagger + \frac{(\tilde{M}_M)_{IJ}}{2} \overline{\Psi_{R_I}^{(0)c}} \Psi_{R_J}^{(0)} \right) + \text{h.c.} \right]. \quad (7.3)$$

Inserting eq. (7.2) into eq. (7.3), one obtains the Majorana mass matrix of the right-handed neutrinos and the neutrino Yukawa matrix in four dimensions as $(M_M)_{IJ} = f_I f_J (\tilde{M}_M)_{IJ}$ and $F_{I\alpha} = f_I \tilde{F}_{I\alpha}$, respectively, where $f_I \equiv \sqrt{2m_{5,I}/((e^{2m_{5,I}\ell} - 1)M)}$. Then, the seesaw formula can be written as $m_\nu = -FM_M^{-1}F^T v^2 = -\tilde{F} \tilde{M}_M^{-1} \tilde{F}^T v^2$. Note that the factors $f_I$ are canceled out in the seesaw formula. This is one of the important properties of the split seesaw mechanism, which results into the following feature: one can realize strongly hierarchical Majorana mass spectra of the right-handed neutrinos and the neutrino Yukawa couplings due to the exponential suppression factor $f_I$ by taking suitable values of $m_{5,I}$ without introducing strongly hierarchical many-mass scales. For example, one can realize a splitting Majorana mass spectrum of the right-handed neutrinos containing both the keV and intermediate mass scales as $(M_{M,1}, M_{M,2}, M_{M,3}) = (5 \text{ keV}, 10^{11} \text{ GeV}, 10^{12} \text{ GeV})$ by taking $(m_{5,1}\ell, m_{5,2}\ell, m_{5,3}\ell) \simeq (23.3, 3.64, 2.26)$, $M = 5 \times 10^{17} \text{ GeV}$, $\ell^{-1} = 10^{16} \text{ GeV}$, and $(\tilde{M}_M)_{II} = 10^{15} \text{ GeV}$. In this example, the mass scales in the model are only the super heavy $(\mathcal{O}(10^{15-17}) \text{ GeV})$ and the electroweak ones, and the small active neutrino mass can
be realized by the (split) seesaw mechanism. Furthermore, the lightest sterile neutrino can be a candidate for DM and the two heavier ones could generate the baryon asymmetry of the Universe via the leptogenesis mechanism.

Next, we comment on the active-sterile mixing angle in the (split) seesaw model. It is well known that the sterile neutrino DM with the keV mass should not contribute to the active neutrino masses through the (split) seesaw mechanism in order to satisfy astrophysical X-ray bounds on the active-sterile mixing angle (equivalent to the corresponding neutrino Yukawa couplings) as $\theta^2 \equiv \sum |F_{1\alpha}|^2 v^2 / M_{M,1}^2 \simeq \mathcal{O}(10^{-10})$ in the few keV region of $M_{M,1}$. This bound implies that when the keV sterile neutrino DM gives the small active neutrino mass through the (split) seesaw, the model is in conflict with the astrophysical bound. The simple reason is that the active-sterile mixing angle becomes large for small masses, $\theta^2 = \sum |F_{1\alpha}|^2 v^2 / M_{M,1}^2 \simeq m_{\text{sol(atm)}/M_{M,1}} \simeq \mathcal{O}(10^{-6} - 10^{-5})$, where $m_{\text{sol(atm)}}$ denote the solar and the atmospheric neutrino mass scales. In other words, while both the Dirac Yukawa coupling and the sterile neutrino mass are suppressed by the size of the extra dimension, the suppression of the coupling enforced by the split seesaw mechanism is weaker; accordingly, active sterile mixing is somewhat enhanced. Thus, an additional suppression for $\theta$ (or, equivalently, $|F_{1\alpha}|$) is required to satisfy the X-ray bound in a realistic model with keV neutrino DM. Such a situation can be easily realized by assuming the Yukawa couplings of the keV sterile neutrino to be additionally small compared with those of the other sterile neutrinos. In this case, the solar and the atmospheric neutrino mass scales are induced from the (split) seesaw with two heavier right-handed neutrinos. On the other hand, the separate seesaw can give an additional suppression factor for the active-sterile mixing.

The separate seesaw model [757] is a variant of the split seesaw. A fundamental assumption of the separate seesaw is that different generations of right-handed neutrinos can have different localization of the five-dimensional wave function and a Majorana mass of the right-handed neutrino unlike the split seesaw. For a realistic model building of keV neutrino DM, we consider the following action of the separate seesaw model,

$$ S = \int d^4 x \int_0^\ell dy \left[ M \left\{ i \overline{\Psi_{R_I}^{(0)}} \Gamma^A \partial_A \Psi_{R_I}^{(0)} + \left( m_{5,1} \overline{\Psi_{R_1}^{(0)}} \Psi_{R_1}^{(0)} - m_{5,J} \overline{\Psi_{R_J}^{(0)}} \Psi_{R_J}^{(0)} \right) \right\} \right. \\ \left. - \left\{ \delta(y) \left( (\tilde{F}^\dagger)_{I\alpha} \overline{\Psi_{R_I}^{(0)}} l_{L\alpha} \tilde{\Phi}^\dagger + \frac{\tilde{M}_{M,1}}{2} \overline{\Psi_{R_1}^{(0)c}} \Psi_{R_1}^{(0)} \right) + \delta(y-\ell) \frac{\tilde{M}_{M,J}}{2} \overline{\Psi_{R_J}^{(0)c}} \Psi_{R_J}^{(0)} + \text{h.c.} \right\} \right] \quad (7.4) $$

where $J = 2,3$ and we take $\tilde{M}_M = \text{diag}\{\tilde{M}_{M,2}, \tilde{M}_{M,3}\}$. Thus, the extra-dimensional wave function profiles of the right-handed neutrinos are given by $\Psi_{R_1}^{(0)} = f_1 e^{m_{5,1}y} \psi_{R_1}^{(0)}(x)$ and $\Psi_{R_J}^{(0)} = g_J e^{-m_{5,J}y} \psi_{R_J}^{(0)}(x)$, where $g_J \equiv f_J e^{m_{5,J}\ell}$. Note that the wave functions of the second and third generations of the right-handed neutrinos localize at the SM brane and their Majorana masses localize at the hidden brane, unlike for split seesaw, while the localization for the first generation is the same as in the split seesaw. As a result, the Majorana masses of the right-handed neutrinos and the neutrino Yukawa coupling matrix in four dimensions are $M_{M,I} = f_I^2 \tilde{M}_{M,I}$, $F_{1\alpha} = f_1 \tilde{F}_{I\alpha}$, and $F_{J\alpha} = g_J \tilde{F}_{J\alpha}$, respectively. The separate seesaw gives the active neutrino mass matrix and a typical mass spectrum as

$$ (m_\nu)_{\alpha\beta} = \left( \tilde{F}_{1\alpha} \tilde{F}_{1\beta} \tilde{M}_{M,1}^{-1} + \sum_{J=2,3} e^{2m_{5,J}\ell} \tilde{F}_{J\alpha} \tilde{F}_{J\beta} \tilde{M}_{M,J}^{-1} \right) v^2, \quad (7.5) $$

$$ m_1 \sim \mathcal{O}(|\tilde{F}_{1\alpha}|^2 v^2 \tilde{M}_{M,1}^{-1}), \quad m_J \sim \mathcal{O}(e^{2m_{5,J}\ell} |\tilde{F}_{J\alpha}|^2 v^2 \tilde{M}_{M,J}^{-1}), \quad (7.6) $$
respectively. This separate seesaw can also realize strongly hierarchical mass spectra of the sterile neutrinos (e.g., ($M_{M,1}$, $M_{M,2}$, $M_{M,3}$) = (5 keV, $10^{11}$ GeV, $10^{12}$ GeV)). Turning to the mass spectrum of the active neutrinos, $m_1$ is suppressed compared to $m_J$, and $m_J$ can realize the solar and the atmospheric neutrino mass scales. This suppression affects on the size of the left-right mixing angle as $\theta^2 = \sum |F_{1\alpha}|^2 v^2 / M_{M,1}^2 \simeq m_{\text{sol(atm)}} / (e^{2m_5 J^\ell} M_{M,1})$. Finally, one can realize $\theta^2 \sim \mathcal{O}(10^{-10})$ without assuming additional suppressions for the Yukawa couplings of only the first generation of sterile neutrinos.

### 7.2.2 Suppressions based on the Froggatt-Nielsen mechanism (authors: A. Merle, V. Niro)

The Froggatt-Nielsen (FN) mechanism [202] has been used in the literature to explain very strong mass hierarchies, for example, in the quark sector. In ref. [758], it was shown that the FN mechanism could explain the existence of one keV scale sterile neutrino, whereas the other two sterile neutrinos have masses at least at GeV scale: $M_1 \simeq \mathcal{O}(\text{keV})$ and $M_2, M_3 \gtrsim \mathcal{O}(\text{GeV})$. The most important point of the work was to find out the minimal assignments for the FN charges to successfully explain the pattern in the sterile neutrino sector, while being in full agreement with the rest of the lepton data. Note that also in ref. [174] a FN mechanism, together with a $Z_3$ symmetry, was used in the context of an $A_4$ flavor symmetry. This model will be discussed in section 7.3.3.

The FN flavon field can acquire a VEV $\langle \Theta \rangle$. Calling $\Lambda$ the cut-off scale at which the heavy sector of the theory is integrated out, the parameter $\lambda = \langle \Theta \rangle / \Lambda$ is a small number of the order of the Cabibbo angle: $\lambda \simeq 0.22$ [759]. After the flavon field acquires a VEV, the corresponding mass matrix of right-handed neutrinos, $M_M$, will have the following structure:

$$ M_M = \begin{pmatrix} \tilde{M}_M^{11} \lambda^{|2g_1|} & \tilde{M}_M^{12} \lambda^{|g_1+g_2|} & \tilde{M}_M^{13} \lambda^{|g_1+g_3|} \\ \bullet & \bullet & \bullet \\ \bullet & \bullet & \tilde{M}_M^{33} \lambda^{|2g_3|} \end{pmatrix}, \qquad (7.7) $$

where $g_i$ are the FN charges of the right-handed neutrinos. Calculating the light neutrino mass matrix, it can be seen that the seesaw formula is not spoiled by the presence of a keV neutrino. Indeed, the $U(1)_{\text{FN}}$ charges of the right-handed fermions cancel out. As a consequence, the basic structure of the neutrino mass matrices is the same for type I and type II seesaw scenarios.

If we suppose that all the coefficients in the matrix $M_M$ are of the same order, then the explanation of a hierarchical spectrum in the sterile neutrino sector should solely come from the FN charges. Considering $g_1 \ge g_2 \ge g_3$, the minimal conditions to be fulfilled are:

$$ \begin{cases} g_1 \ge g_1|_{\text{min}} & \text{with } g_1|_{\text{min}} = g_2 + 3, \\ g_2 \ge g_2|_{\text{min}} & \text{with } g_2|_{\text{min}} = g_3. \end{cases} $$

In figure 29, we have schematically depicted the general effect of the FN mechanism, where a certain mass scale $M_0$ is multiplied by powers of $\lambda$ that depend on the fermion generation. A reasonable condition would be to set $g_3 = 0$ and $g_1 = g_2 + 3$. Two minimal scenarios are given by $(g_1, g_2, g_3) = (3, 0, 0)$ and $(g_1, g_2, g_3) = (4, 1, 0)$. In the following, these will be called Scenario A and Scenario B, respectively.

The way the FN charged are combined considerably restricts the allowed parameter space. Indeed, despite the fact that the FN charge assignment seems to involve great freedom,
<figure><img src="image_30.png" /><figcaption>**Figure 29.** The mass shifting scheme of Froggatt-Nielsen models. (Figure similar to figure 2 in ref. [758] and to figure 5 in ref. [66].)</figcaption></figure>

several scenarios are not able to successfully produce a light neutrino sector compatible with experimental data. In particular, the FN scenarios disagree with the Left-Right symmetric framework for keV sterile neutrino DM. It turns out that an ideal framework is represented by an SU(5) inspired model with two FN fields, see the detailed discussion in ref. [758].

A problem of models with only one FN field is that they normally lead to small atmospheric neutrino mixing and are thus incompatible with the data, see discussion in refs. [760–762]. The only way out would be to consider a pseudo-Dirac scenario and, thus, set some elements of the mass matrices equal to zero [763]. However, this would be a specific choice, while the most general approach is to consider two FN fields. The model presented in refs. [760, 761] contains two FN flavon fields $\Theta_{1,2}$ which can obtain complex VEVs:

$$ \lambda = \frac{\langle \Theta_1 \rangle}{\Lambda}, \quad R = \frac{\langle \Theta_1 \rangle}{\langle \Theta_2 \rangle} = R_0 e^{i\alpha_0}, \qquad (7.8) $$

with $R_0$ and $\alpha_0$ being real numbers. Besides having two FN fields, it is also necessary to introduce an auxiliary $\mathbb{Z}_2$ symmetry, such that the phase $\alpha_0$ can be responsible for CP violation [761]. The most general Lagrangian in a seesaw type II scenario is then given by

$$ \mathcal{L} = - \sum_{a,b,\alpha,\beta}^{a+b=k_\alpha+f_\beta} Y_e^{\alpha\beta} \overline{e_{\alpha R}} H L_{\beta L} \lambda_1^a \lambda_2^b + \text{h.c.} - \sum_{a,b,I,\alpha}^{a+b=g_I+f_\alpha} Y_D^{I\alpha} \overline{\nu_{IR}} \tilde{H} L_{\alpha L} \lambda_1^a \lambda_2^b + \text{h.c.} \quad (7.9) \\ - \sum_{a,b,\alpha,\beta}^{a+b=f_\alpha+f_\beta} \frac{1}{2} (L_{\alpha L})^C \tilde{m}_L^{\alpha\beta} L_{\beta L} \lambda_1^a \lambda_2^b + \text{h.c.} - \sum_{a,b,I,J}^{a+b=g_I+g_J} \frac{1}{2} \overline{\nu_{IR}} \tilde{M}_M^{IJ} (\nu_{JR})^C \lambda_1^a \lambda_2^b + \text{h.c.}, $$

where, by using eq. (7.8), we have defined

$$ \lambda_1^a \lambda_2^b \equiv \left(\frac{\Theta_1}{\Lambda}\right)^a \left(\frac{\Theta_2}{\Lambda}\right)^b = \lambda^{a+b} R^b. \qquad (7.10) $$

The matrices $Y_e, Y_D, \tilde{M}_M$ and $\tilde{m}_L$ are the charged lepton Yukawa matrix, the Dirac neutrino Yukawa matrix, the uncorrected right-handed Majorana neutrino mass matrix, and the uncorrected left-handed Majorana neutrino mass matrix. Note that the matrix elements of $\tilde{M}_M$ and $\tilde{m}_L$ are all of the same order, since they are not yet corrected by FN contributions. Moreover, certain terms in the Lagrangian of eq. (7.10) might violate the $\mathbb{Z}_2$ parity and must be set to zero. We adopt the following FN charge and $\mathbb{Z}_2$ assignments inspired by
refs. [761, 764], for the FN fields, for the lepton doublets, and for the right-handed charged leptons and neutrinos:

$$ \begin{aligned} \Theta_{1,2} & : (-1, -1; +, -), & L_{1,2,3} & : (a + 1, a, a; +, +, -), \\ \bar{e}_{1,2,3} & : (3, 2, 0; +, +, -), & \bar{\nu}_{1,2,3R} & : (g_1, g_2, g_3; +, +, -), \end{aligned} \qquad (7.11) $$

where $a = 0, 1$. Considering this assignment, the mass eigenvalues for the right-handed neutrinos as functions of the right-handed mass scale $M_0$ are:

$$ \text{Scenario A:} \quad M_1 = M_0 \lambda^6 2 R_0^2 \sqrt{1 + R_0^4 + 2 R_0^2 \cos(2\alpha_0)}, \quad M_2 = M_0, \quad M_3 \approx M_0, $$

$$ \text{Scenario B:} \quad M_1 = M_0 \lambda^8 2 R_0^4 \sqrt{1 + R_0^8 - 2 R_0^4 \cos(4\alpha_0)}, \quad M_2 = M_0 \lambda^2, \quad M_3 \approx M_0. $$

If we consider $M_1$ of $\mathcal{O}(\text{keV})$, then $M_0$ is about $10^6 \text{ keV} \sim 1 \text{ GeV}$ for Scenario A, or about $10^8 \text{ keV} \sim 100 \text{ GeV}$ for Scenario B.

Fully democratic matrices in SU(5) inspired FN models are, in general, not in agreement with the experimental constraints on the leptonic mixing angles, as has already been discussed in ref. [765]. However, the agreement with the data strongly improves considering slightly non-democratic matrices. Considering, for example, the Dirac Yukawa couplings $Y_D^{12} = Y_D^{22} = \delta_0 y_D$, instead of $Y_D^{12} = Y_D^{22} = y_D$, is possible to increase the consistency with light neutrino data. In order to find fully working models, however, a numerical analysis is necessary, see ref. [758] for details.

### 7.2.3 The minimal radiative inverse seesaw mechanism (authors: A. Pilaftsis, B. Dev)

A technically natural realization for the seesaw scale to be in the TeV range is the so-called Inverse Seesaw Model (ISM) [237, 766, 767], where in addition to RH neutrinos {$\nu_{R,I}$}, another set of SM singlet fermions {$S_{L,a}$} are introduced. After electroweak symmetry breaking, the neutrino Yukawa sector of a general ISM is described by the Lagrangian

$$ \mathcal{L}_Y = -\bar{\nu}_L m_D \nu_R - \bar{S}_L m_N \nu_R - \frac{1}{2} \bar{\nu}_R^c M_M \nu_R - \frac{1}{2} \bar{S}_L \mu_S S_L^c + \text{h.c.} \qquad (7.12) $$

For a symmetric extension of the SM with three pairs of singlet neutrinos, eq. (7.12) gives rise to the following 9 × 9 neutrino mass matrix in the basis {$\langle \nu_{L,\alpha} \rangle^c, \nu_{R,I}, (S_{L,a})^c$}:

$$ \mathcal{M}_{\nu} = \begin{pmatrix} 0 & m_D & 0 \\ m_D^T & M_M & m_N^T \\ 0 & m_N & \mu_S \end{pmatrix}. \qquad (7.13) $$

Here we have not included in $\mathcal{L}_Y$ the dimension-four lepton-number breaking term $\bar{l}_L \Phi S_L^c$ which appears, for instance, in linear seesaw models [205, 238, 768], since the resulting neutrino mass matrix in presence of this term can always be rotated to the form given in (7.13) [769]. In the limit of $M_M, \mu_S \to 0$, lepton number symmetry is restored and the light neutrinos $\nu_{L,\alpha}$ are massless to all orders in perturbation, whereas {$\nu_{R,I}, S_{L,a}$} form three SM-singlet massive Dirac neutrinos.

In the original ISM presented in ref. [237], the RH neutrino Majorana mass is zero, $M_M = 0$ in (7.13). In this case, for $|\mu_S| \ll |m_N|, |m_D|$, it is possible to have very light sterile
neutrinos, e.g. in the presence of a $\mu - \tau$ symmetry [770] or in theories with warped extra dimensions [771].

Another interesting realization of ISM arises when $M_M \neq 0$ but $\mu_S = 0$ [772]. This may occur, for instance, in models where the Majorana masses for the S-fields are forbidden due to specific flavor symmetries [174, 773–775]. In this case, the rank of the mass matrix given by (7.13) reduces to six and the light neutrinos are *exactly* massless at tree-level. However, they acquire a non-zero mass at the one-loop level which is *directly* proportional to the Majorana mass matrix $M_M$ and results from standard electroweak radiative corrections involving the neutral gauge and Higgs bosons [776]. This scenario is known as the Minimal Radiative Inverse Seesaw Model (MRISM) [772] which is very economical, since it does not require the existence of other non-standard scalar or gauge fields or other fermionic matter beyond the singlet neutrinos {$\nu_{R,I}, S_{L,a}$} necessary for a general ISM.

The possibility of having light sterile neutrinos in the MRISM can be realized in the limit $|M_M| \gg |m_D|, |m_N|$ [777]. In this limit, the $\nu_{R,I}$-fields decouple below the mass scale $M_M$, resulting in an effective theory with six neutrino states: $\nu_{L,\alpha}$ and $S_{L,a}$. At tree-level, the effective 6 × 6 neutrino mass matrix in the weak basis becomes

$$ \mathcal{M}_{\text{eff}}^{\text{tree}} = \begin{pmatrix} m_D \\ m_N \end{pmatrix} \mathcal{M}_M^{-1} \begin{pmatrix} m_D^T & m_N^T \end{pmatrix} = \begin{pmatrix} m_D M_M^{-1} m_D^T & m_D M_M^{-1} m_N^T \\ m_N M_M^{-1} m_D^T & m_N M_M^{-1} m_N^T \end{pmatrix}. \quad (7.14) $$

Note that one of the block eigenvalues must vanish, since the effective mass matrix $\mathcal{M}_{\text{eff}}^{\text{tree}}$ is of rank 3.

At one-loop level, the mass matrix given by eq. (7.14) receives an electroweak radiative correction proportional to $M_M$ [772]:

$$ M_{\nu_L}^{1-\text{loop}} = \frac{\alpha_w}{16\pi m_W^2} \mathcal{M}_D \mathcal{M}_S \left[ m_H^2 (\mathcal{M}_S^2 - m_H^2 \mathbf{1})^{-1} \ln \left( \frac{\mathcal{M}_S^2}{m_H^2} \right) + 3m_Z^2 (\mathcal{M}_S^2 - m_Z^2 \mathbf{1})^{-1} \ln \left( \frac{\mathcal{M}_S^2}{m_Z^2} \right) \right] \mathcal{M}_D^T, \quad (7.15) $$

where $\alpha_w \equiv g^2/(4\pi)$ is the weak coupling strength, $m_{H,W,Z}$ are the masses of the SM Higgs, W and Z bosons respectively, and we have defined a 3 × 6 mass matrix $\mathcal{M}_D = (m_D, 0)$ and a 6 × 6 mass matrix $\mathcal{M}_S = \begin{pmatrix} M_M & m_N^T \\ m_N & \mu_S \end{pmatrix}$. In the limit $|M_M| \gg |m_D|, |m_N|$, eq. (7.15) simplifies to [777]

$$ \mathcal{M}_{\text{eff}}^{1-\text{loop}} \approx \begin{pmatrix} m_D M_M^{-1} x_R f(x_R) m_D^T & 0 \\ 0 & 0 \end{pmatrix} \equiv \begin{pmatrix} \Delta M & 0 \\ 0 & 0 \end{pmatrix}, \quad (7.16) $$

where the one-loop function $f(x_R)$ is defined as

$$ f(x_R) = \frac{\alpha_W}{16\pi} \left[ \frac{x_H}{x_R - x_H} \ln\left(\frac{x_R}{x_H}\right) + \frac{3x_Z}{x_R - x_Z} \ln\left(\frac{x_R}{x_Z}\right) \right], \quad (7.17) $$

with $x_R \equiv \hat{M}_M^2/m_W^2$, $x_H \equiv m_H^2/m_W^2$, $x_Z \equiv m_Z^2/m_W^2$, assuming $M_M = \hat{M}_M \mathbf{1}$ for simplicity. Thus, the full effective neutrino mass matrix in the basis {$\nu_{L,\alpha}, S_{L,a}$} is given by

$$ \mathcal{M}_{\text{eff}} = \mathcal{M}_{\text{eff}}^{\text{tree}} + \mathcal{M}_{\text{eff}}^{1-\text{loop}} = \begin{pmatrix} m_D M_M^{-1} (\mathbf{1} + x_R f(x_R)) m_D^T & m_D M_M^{-1} m_N^T \\ m_N M_M^{-1} m_D^T & m_N M_M^{-1} m_N^T \end{pmatrix}. \quad (7.18) $$
This can be diagonalized by a general 6 × 6 unitary matrix parametrized in terms of 15 Euler angles and 10 Dirac phases. Of the resulting six light mass eigenstates, three should describe the mainly active neutrinos and the remaining three will be predominantly sterile states. In ref. [777], it was shown that one of the sterile states could be at the keV-scale, having very small mixing ≲ 10⁻⁵ with the active neutrinos, thus serving as a potentially WDM candidate. Depending on the masses of the other two sterile states, one could consider several possibilities, e.g. (i) if both of them are in the eV range, possessing a non-zero mixing with the active states, one could explain the LSND+MiniBooNE+reactor neutrino data, or (ii) if one of the light sterile states is in the eV range and the second one is superlight and almost mass-degenerate with the solar neutrinos, it could also give rise to potentially observable effects in solar neutrino experiments [778–780]. Moreover, this scenario is more compatible with the cosmological constraints on the sum of the neutrino masses.

Thus, the MRISM provides a minimal extension of the SM accommodating neutrino masses and keV-scale sterile neutrino Dark Matter, while being consistent with all the existing experiments, including the anomalies in the neutrino oscillation data. In addition, the heavy singlet neutrinos in this model with a degenerate mass spectrum could be used to explain the observed matter-antimatter asymmetry in the Universe through the mechanism of resonant leptogenesis [393, 781, 782].

### 7.2.4 Models based on loop-suppressions (authors: D. Borah, R. Adhikari)

Tiny neutrino masses can also arise at one-loop level within the class of the so-called “scotogenic” models first proposed by Ma [214]. These models can not only motivate tiny neutrino masses at one-loop level [783–785], but they can also give rise to at least one stable DM candidate [30, 31, 783, 784, 786] — and in general, the fact that the neutrino mass is generated at loop-level strongly affects their phenomenology [783, 787–794]. An Abelian gauge extended version of the original scotogenic model was proposed in [795] to explain tiny active neutrino masses as well as DM. Suitable modifications of this model accommodating tiny sterile neutrino masses have appeared in [796–798]. Models with dynamical symmetry breaking, loop suppression of neutrino masses, and a resultant low-scale seesaw have been studied in [220–222].

The field content of the Abelian gauge model [795] is shown in table 11. The third column shows the quantum numbers of the fields under a new U(1)$_X$ gauge symmetry. The fourth column indicates the transformations under the remnant $\mathbb{Z}_2$ symmetry, such that the lightest $\mathbb{Z}_2$-odd particle is stable. The combination of one singlet fermion $\nu_R$ and two triplet fermions $\Sigma_{1R}, \Sigma_{2R}$ with suitable Higgs scalars is chosen such that tiny neutrino masses arise at one-loop level. Two more singlets $S_{1R}, S_{2R}$ are required to be present in order to satisfy the anomaly freedom conditions. The Higgs doublets $\Phi_1, \Phi_2$ give masses to quarks and charged leptons, respectively. The Yukawa Lagrangian relevant for the neutrino mass is

$$ \mathcal{L}_Y \supset y \bar{L} \Phi_1^\dagger S_{1R} + h_N \bar{L} \Phi_3^\dagger \nu_R + h_\Sigma \bar{L} \Phi_3^\dagger \Sigma_R + f_N \nu_R \nu_R \chi_4 + f_S S_{1R} S_{1R} \chi_1 \\ + f_\Sigma \Sigma_R \Sigma_R \chi_4 + f_{NS} \nu_R S_{2R} \chi_2^\dagger + f_{12} S_{1R} S_{2R} \chi_3^\dagger, \qquad (7.19) $$

with VEVs $\langle \phi_{1,2}^0 \rangle = v_{1,2}, \langle \chi_{1,4}^0 \rangle = u_{1,4}$. The tree-level neutrino mass from eq. (7.19) is

$$ m_{\nu 3} \approx \frac{2y^2 v_1^2}{f_S u_1}, \qquad (7.20) $$
<table><caption>**Table 11.** Particle content of the model.</caption><thead><tr><th>Particle</th><th>SU(3)<sub>c</sub> &times; SU(2)<sub>L</sub> &times; U(1)<sub>Y</sub></th><th>U(1)<sub>X</sub></th><th>Z<sub>2</sub></th></tr></thead><tbody><tr><td>(u, d)<sub>L</sub></td><td>(3, 2, <sup>1</sup>&frasl;<sub>6</sub>)</td><td>n<sub>1</sub></td><td>+</td></tr><tr><td>u<sub>R</sub></td><td>(3&#773;, 1, <sup>2</sup>&frasl;<sub>3</sub>)</td><td><sup>1</sup>&frasl;<sub>4</sub>(7n<sub>1</sub> &minus; 3n<sub>4</sub>)</td><td>+</td></tr><tr><td>d<sub>R</sub></td><td>(3&#773;, 1, &minus;<sup>1</sup>&frasl;<sub>3</sub>)</td><td><sup>1</sup>&frasl;<sub>4</sub>(n<sub>1</sub> + 3n<sub>4</sub>)</td><td>+</td></tr><tr><td>(&nu;, e)<sub>L</sub></td><td>(1, 2, &minus;<sup>1</sup>&frasl;<sub>2</sub>)</td><td>n<sub>4</sub></td><td>+</td></tr><tr><td>e<sub>R</sub></td><td>(1, 1, &minus;1)</td><td><sup>1</sup>&frasl;<sub>4</sub>(&minus;9n<sub>1</sub> + 5n<sub>4</sub>)</td><td>+</td></tr><tr><td>&nu;<sub>R</sub></td><td>(1, 1, 0)</td><td><sup>3</sup>&frasl;<sub>8</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>-</td></tr><tr><td>&Sigma;<sub>1R,2R</sub></td><td>(1, 3, 0)</td><td><sup>3</sup>&frasl;<sub>8</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>-</td></tr><tr><td>S<sub>1R</sub></td><td>(1, 1, 0)</td><td><sup>1</sup>&frasl;<sub>4</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>+</td></tr><tr><td>S<sub>2R</sub></td><td>(1, 1, 0)</td><td>&minus;<sup>5</sup>&frasl;<sub>8</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>-</td></tr><tr><td>(&phi;<sup>+</sup>, &phi;<sup>0</sup>)<sub>1</sub></td><td>(1, 2, &minus;<sup>1</sup>&frasl;<sub>2</sub>)</td><td><sup>3</sup>&frasl;<sub>4</sub>(n<sub>1</sub> &minus; n<sub>4</sub>)</td><td>+</td></tr><tr><td>(&phi;<sup>+</sup>, &phi;<sup>0</sup>)<sub>2</sub></td><td>(1, 2, &minus;<sup>1</sup>&frasl;<sub>2</sub>)</td><td><sup>1</sup>&frasl;<sub>4</sub>(9n<sub>1</sub> &minus; n<sub>4</sub>)</td><td>+</td></tr><tr><td>(&phi;<sup>+</sup>, &phi;<sup>0</sup>)<sub>3</sub></td><td>(1, 2, &minus;<sup>1</sup>&frasl;<sub>2</sub>)</td><td><sup>1</sup>&frasl;<sub>8</sub>(9n<sub>1</sub> &minus; 5n<sub>4</sub>)</td><td>-</td></tr><tr><td>&chi;<sub>1</sub></td><td>(1, 1, 0)</td><td>&minus;<sup>1</sup>&frasl;<sub>2</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>+</td></tr><tr><td>&chi;<sub>2</sub></td><td>(1, 1, 0)</td><td>&minus;<sup>1</sup>&frasl;<sub>4</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>+</td></tr><tr><td>&chi;<sub>3</sub></td><td>(1, 1, 0)</td><td>&minus;<sup>3</sup>&frasl;<sub>8</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>-</td></tr><tr><td>&chi;<sub>4</sub></td><td>(1, 1, 0)</td><td>&minus;<sup>3</sup>&frasl;<sub>4</sub>(3n<sub>1</sub> + n<sub>4</sub>)</td><td>+</td></tr></tbody></table>

<figure><img src="image_31.png" /><figcaption>**Figure 30.** One-loop contribution to active neutrino mass. The figure is inspired from ref. [799].</figcaption></figure>

while the other two active neutrinos are massless and only become massive at one-loop, see figure 30 and ref. [799] for details. The sub-eV scale for active neutrino masses is generated naturally by adjusting the Yukawa couplings involved in figure 30.

The model shown in table 11 has three sterile neutrinos $S_{1R}$, $S_{2R}$, $\nu_R$ out of which $S_{1R}$ couples to the active neutrinos and contributes to the tree-level mass term of $\nu_3$. Thus, for generic Dirac Yukawa couplings of the neutrinos, the sterile neutrino $S_{1R}$ is expected to be much heavier than the eV scale. As seen from the Lagrangian in eq. (7.19), $S_{1R}$ and $\nu_R$ acquire non-zero masses from the VEVs of $\chi_1$, $\chi_4$, respectively. Since the Abelian gauge symmetry is spontaneously broken down to the standard model by the VEVs of $\chi_1$, $\chi_4$ at
<figure><img src="image_32.png" /><figcaption>**Figure 31.** One-loop contributions to the sterile neutrino mass (first and second diagram) and to active-sterile mixing (third diagram). Figures are inspired from [796, 797].</figcaption></figure>

a scale above the electroweak scale, one can take the VEVs to be at least at the TeV scale. Thus, for generic $O(1)$ Yukawa couplings, the masses of $S_{1R}$, $\nu_R$ also lie at the TeV scale. Unlike for $S_{1R}$ and $\nu_R$, there is no tree-level mass term for the singlet fermion $S_{2R}$. However, there are mixing terms of $S_{2R}$ with $\nu_R$, $S_{1R}$ through the Higgs fields $\chi_2$, $\chi_3$ respectively. If these Higgs fields do not acquire non-zero VEVs, then $S_{2R}$ remains massless at tree-level. Thus, the tree-level neutrino mass matrix in the basis $(\nu_e, \nu_\mu, \nu_\tau, S_{1R}, S_{2R}, \nu_R)$ is

$$ M_f = \begin{pmatrix} 0 & 0 & 0 & y_1 v_1 & 0 & 0 \\ 0 & 0 & 0 & y_2 v_2 & 0 & 0 \\ 0 & 0 & 0 & y_3 v_3 & 0 & 0 \\ y_1 v_1 & y_2 v_2 & y_3 v_3 & f_S u_1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 & f_N u_4 \end{pmatrix}. \qquad (7.21) $$

The sterile neutrino $S_{2R}$, which remains massless at tree-level, can acquire a small eV scale mass at one-loop level from the diagrams shown in figure 31, which corresponds to a non-zero $(M_f)_{55}$ element in the above basis. With suitable choices of the couplings and the VEVs, it is possible to get the desired loop-suppression for the mass of the sterile neutrino, no matter if eV or keV.

Apart from the mass for sterile neutrinos, one should also address the issue of their mixing with active neutrinos. In particular astrophysical observations can constrain the mixing of keV neutrinos with active neutrinos. In the original version of this model [795] with the minimal field content, there is no mixing between active and sterile neutrinos. In the later work [796], the field content was extended by a new scalar singlet field $\zeta$ with $U(1)_X$ charge $\frac{5}{8}(3n_1+n_4)$, which allows for a tree-level mixing term of $N_R$ and $S_{1R}$ and also for one-loop mixing between active and sterile neutrinos, see the third diagram shown in figure 31.

In the original model [795], the singlet fermion $S_{2R}$, which was taken to be the light sterile neutrino in [796], is $\mathbb{Z}_2$-odd. The one-loop mass of this sterile neutrino can be adjusted at eV or keV scale, with a mixing that vanishes at leading order. However, allowing for mixing between the $\mathbb{Z}_2$-odd singlet fermion $S_{2R}$ with the $\mathbb{Z}_2$-even active neutrinos breaks the $\mathbb{Z}_2$ symmetry, making the sterile neutrino unstable. This motivates $S_{2R}$ as decaying keV sterile neutrino DM.
<table><caption>**Table 12.** $\mathcal{F} \equiv L_e - L_\mu - L_\tau$ charges, where $L_{\alpha L} = (\nu_{\alpha L}, \alpha_L)^T$, and $\alpha = e, \mu, \tau$. The scalars are defined as: $\phi = (\phi^+, \phi^0)^T$, $\Delta = ((\Delta^+/\sqrt{2}, \Delta^{++}), (\Delta^0, -\Delta^+/\sqrt{2}))$.</caption><thead><tr><th>Fields</th><th>L<sub>eL</sub></th><th>L<sub>&mu;L</sub></th><th>L<sub>&tau;L</sub></th><th>e<sub>R</sub></th><th>&mu;<sub>R</sub></th><th>&tau;<sub>R</sub></th><th>&nu;<sub>1R</sub></th><th>&nu;<sub>2R</sub></th><th>&nu;<sub>3R</sub></th><th>&phi;</th><th>&Delta;</th></tr></thead><tbody><tr><td>F</td><td>1</td><td>-1</td><td>-1</td><td>1</td><td>-1</td><td>-1</td><td>1</td><td>-1</td><td>-1</td><td>0</td><td>0</td></tr></tbody></table>

Finally, let us remark that further modifications of the model can lead to even richer phenomenology, such as multi-component DM with a mixed WDM/CDM setting [797].

## 7.3 Models based on symmetry breaking

The next class of mechanisms are those based on some type of flavor symmetry. These symmetries can either be used to force one sterile neutrino to be massless at leading order – which is then corrected at next-to-leading order to a finite value – or they can complement a suppression mechanism already present in the model by enforcing the mixing angle between active and sterile neutrinos to be small enough in order not to contradict observations.

### 7.3.1 $L_e - L_\mu - L_\tau$ symmetry (authors: A. Merle, V. Niro)

In refs. [239, 800, 801], a $L_e - L_\mu - L_\tau$ flavor symmetry was applied at the same time to active and sterile neutrinos. If the symmetry is unbroken, the model predicts a massless neutrino and two neutrinos with exactly the same mass, both in the active and in the sterile sectors. For this reason, the $L_e - L_\mu - L_\tau$ flavor symmetry should be softly broken. This leads to a non-zero mass for one of the active and one of the sterile neutrinos. The other four neutrinos (two active and two sterile), after the introduction of soft-breaking terms in the Lagrangian, are not anymore degenerate in mass. This model could naturally accomodate for a keV sterile neutrino in the spectrum, since the soft symmetry breaking terms will be much smaller than the masses of the almost degenerate sterile neutrinos. Note that an important feature of this model is the prediction of a strongly hierarchical neutrino spectra.

Let us denote the symmetry as $\mathcal{F} \equiv L_e - L_\mu - L_\tau$. The particle content and the charge assignments of our model are given in table 12. We extended the model previously proposed in ref. [802] by a Higgs triplet $\Delta$, in order to accommodate a type II seesaw, which is present, for example, in the context of a left-right symmetric framework, like the one presented in ref. [671], required also for a sufficient entropy production. Considering the limit of vanishing left-handed mass matrix and/or triplet Yukawa coupling, the model can also be applied to scenarios where the Higgs triplet is not present.

The full neutrino mass term is given by

$$ \mathcal{L}_{\text{mass}} = -\frac{1}{2} \overline{\Psi^C} \mathcal{M}_{\nu} \Psi + \text{h.c.}, \quad \text{with} \quad \mathcal{M}_{\nu} = \begin{pmatrix} 0 & m_L^{e\mu} & m_L^{e\tau} & m_D^{e1} & 0 & 0 \\ m_L^{e\mu} & 0 & 0 & 0 & m_D^{\mu2} & m_D^{\mu3} \\ m_L^{e\tau} & 0 & 0 & 0 & m_D^{\tau2} & m_D^{\tau3} \\ m_D^{e1} & 0 & 0 & 0 & M_M^{12} & M_M^{13} \\ 0 & m_D^{\mu2} & m_D^{\tau2} & M_M^{12} & 0 & 0 \\ 0 & m_D^{\mu3} & m_D^{\tau3} & M_M^{13} & 0 & 0 \end{pmatrix}, \quad (7.22) $$

and $\Psi \equiv ((\nu_{eL})^C, (\nu_{\mu L})^C, (\nu_{\tau L})^C, \nu_{1R}, \nu_{2R}, \nu_{3R})^T$. We have defined $m_D^{\alpha i} \equiv v_\phi Y_D^{\alpha i}$ and $m_L^{\alpha \beta} \equiv v_\Delta Y_L^{\alpha \beta}$, with $v_\phi$ and $v_\Delta$ being the vacuum expectation values of the scalar doublet $\phi$ and the scalar triplet $\Delta$.
<figure><img src="image_33.png" /><figcaption>**Figure 32.** The mass shifting schemes due to soft breaking of a global $L_e - L_\mu - L_\tau$ flavor symmetry. (Figure similar to figure 6 of ref. [66] and figure 1 of ref [758].)</figcaption></figure>

The exact symmetry predicts a bimaximal mixing in the neutrino sector:

$$ U_{\nu} = \begin{pmatrix} \frac{1}{\sqrt{2}} & \frac{1}{\sqrt{2}} & 0 \\ -\frac{1}{2} & \frac{1}{2} & \frac{1}{\sqrt{2}} \\ \frac{1}{2} & -\frac{1}{2} & \frac{1}{\sqrt{2}} \end{pmatrix}. \qquad (7.23) $$

This matrix predicts a zero $\theta_{13}$ and a maximal mixing angle $\theta_{12}$, which are both ruled out experimentally. However, it is still possible to obtain a PMNS matrix $U_{\text{PMNS}} = U_L^\dagger U_\nu$ compatible with experimental data, depending on the actual form of the charged lepton mixing matrix $U_L$. Considering CP conservation and a hierarchical relation between $\lambda_{ij} \equiv \sin \theta'_{ij}$, of the type $\lambda_{12} = \lambda$, $\lambda_{13} \simeq \lambda^3$, $\lambda_{23} \simeq \lambda^2$, with $\lambda \simeq 0.20$ being the parameter that describes the deviation of $\theta_{12}$ from $\pi/4$ [803], we find

$$ U_L = \begin{pmatrix} 1 - \lambda^2/2 & \lambda & \lambda^3 \\ -\lambda & 1 - \lambda^2/2 & \lambda^2 \\ \lambda^3 & -\lambda^2 & 1 \end{pmatrix} + \mathcal{O}(\lambda^4). \qquad (7.24) $$

This form of $U_L$, combined with the bimaximal matrix from eq. (7.23), could lead to a $U_{\text{PMNS}}$ matrix compatible with the experimental values [800], as it was also shown in ref. [803]. The matrix $U_L$ is associated to the diagonalization of the matrix

$$ M_l M_l^\dagger \approx \begin{pmatrix} m_e^2 + m_\mu^2 \lambda^2 & m_\mu^2 \lambda & 0 \\ m_\mu^2 \lambda & m_\mu^2 & 0 \\ 0 & 0 & m_\tau^2 \end{pmatrix} + \mathcal{O}(\lambda^3), \qquad (7.25) $$

with $M_l$ being the charged leptons mass matrix.

As stated before, in this scenario we need soft breaking terms to avoid the presence of two zero eigenvalues. For this reason, we add to the neutrino mass in eq. (7.23), a diagonal part containing symmetry-breaking terms:

$$ M_{\nu} \rightarrow M_{\nu} + \text{diag}(s_{L}^{ee}, s_{L}^{\mu\mu}, s_{L}^{\tau\tau}, S_{R}^{11}, S_{R}^{22}, S_{R}^{33}). \qquad (7.26) $$

In the case in which $m_L^{\alpha\beta} \ll m_D^{\alpha I} \ll M_M^{IJ}$ we can block-diagonalize the matrix $M_\nu$, like in the usual type II scenario. Considering an additional unbroken extra $\mu - \tau$ symmetry [804],
we have $M_M^{12} \simeq M_M^{13} \sim M_M$ and $m_L^{e\mu} \simeq m_L^{e\tau} \sim m_L$, and setting, for simplicity, $m_D^{\alpha I} \simeq m_D$, $s_L^{\alpha\alpha} \simeq s$, $S_R^{II} \simeq S$, we obtain the eigenvalues $\mathcal{E}' = \{\lambda'_+, \lambda'_-, \lambda_s, \Lambda'_+, \Lambda'_-, \Lambda_s\}$, with $\lambda'_\pm \simeq \pm\sqrt{2}[m_L - \frac{m_D^2}{M_M}]$, $\lambda_s = s$, $\Lambda'_\pm = S \pm \sqrt{2}M_M$, and $\Lambda_s = S$. In this case, we could explain the presence of one keV sterile neutrino ($S \simeq$ keV) and two heavier sterile neutrinos ($M_M \gg S$) and the neutrino mixing matrix, even in the presence of soft-breaking terms, results compatible with the bimaximal form.

The form of the mass matrix $\mathcal{M}_l$ for the charged leptons will also experience restrictions by the $\mathcal{F}$-symmetry. We can consider soft breaking terms, $s_D^{\alpha I} = v_\phi Y_D^{\alpha I}$ with $\alpha = e, \mu, \tau$ and $I = 1, 2, 3$, and assume, for simplicity, a symmetric form of the charged lepton mass matrix ($s_D^{\tau e} = s_D^{\tau \mu}$, $s_D^{\mu e} = s_D^{\mu \tau}$, and $m_D^{\tau \mu} = m_D^{\mu \tau}$). Under the assumptions of small $s_D^{\tau e}$ and $m_D^{\mu \tau}$, we obtain

$$ \mathcal{M}_l \mathcal{M}_l^\dagger = \begin{pmatrix} (m_D^{ee})^2 + (s_D^{e\mu})^2 & s_D^{e\mu} (m_D^{ee} + m_D^{\mu\mu}) & 0 \\ s_D^{e\mu} (m_D^{ee} + m_D^{\mu\mu}) & (m_D^{\mu\mu})^2 + (s_D^{e\mu})^2 & 0 \\ 0 & 0 & (m_D^{\tau\tau})^2 \end{pmatrix}. \quad (7.27) $$

If we identify $s_D^{e\mu} = m_D^{\mu\mu}\lambda$, $m_D^{ee} = m_e$, $m_D^{\mu\mu} = m_\mu$, and $m_D^{\tau\tau} = m_\tau$, we get a matrix similar to eq. (7.25) and a charged lepton mixing matrix like the one in eq. (7.24). Thus, in the case of softly broken $\mathcal{F}$-symmetry, it is possible to obtain a PMNS matrix that is compatible with experimental data [800]. The model based on a $\mathcal{F}$-symmetry is then a successful model that can explain a hierarchical spectrum in the sterile neutrino sector and, at the same time, reproduce the light neutrino experimental data and even give testable predictions in the form of neutrino mass sum rules [225, 226, 805].

As a final note, the mixing angle in this type of models will scale with the parameter $m_D^{e1}$ in the mass matrix, devided by the keV-scale mass. Thus, the X-ray bound for sterile neutrino DM does pose a constraint on $m_D^{e1}$.

### 7.3.2 $Q_6$ symmetry (author: T. Araki)

We address the origin of the small active-sterile mixing and the mass hierarchy of the sterile neutrinos ($M_1 \ll M_{2,3}$) by exploiting a non-Abelian discrete flavor symmetry. We also demand that masses of the heavier sterile neutrinos are quasi-degenerate ($M_2 \simeq M_3$), which may be desirable for successful baryogenesis. The mass degeneracy of the heavier sterile neutrinos could easily be realized by embedding them into a doublet representation of a given flavor symmetry, while it may be reasonable to assign a singlet representation to the lightest one. Furthermore, if the singlet representation is complex, one can prohibit a bare mass term for the lightest sterile neutrino because of its Majorana nature. Then, a small $M_1$ and a slight mass difference between $M_2$ and $M_3$ would arise after the flavor symmetry is spontaneously broken. In view of these facts, we employ $Q_6$ as our flavor symmetry since it is the smallest discrete group which contains both complex singlet and a real doublet representations.

$Q_6$ consists of four singlet and two doublet irreducible representations: $1, 1', 1'', 1'''$, $2, 2'$, where $1, 1'$ and $2'$ are real, while $2$ and $1''' = (1'')^*$ are complex. The multiplication rules of the irreducible representations are summarized in table 13. Especially,
<table><caption>**Table 13.** The multiplication rules of $Q_6$.</caption><thead><tr><th>$Q_6$</th><th>$1'$</th><th>$1''$</th><th>$1'''$</th><th>$2$</th><th>$2'$</th></tr></thead><tbody><tr><td>$1'$</td><td>$1$</td><td>$1'''$</td><td>$1''$</td><td>$2$</td><td>$2'$</td></tr><tr><td>$1''$</td><td>$1'''$</td><td>$1'$</td><td>$1$</td><td>$2'$</td><td>$2$</td></tr><tr><td>$1'''$</td><td>$1''$</td><td>$1$</td><td>$1'$</td><td>$2'$</td><td>$2$</td></tr><tr><td>$2$</td><td>$2$</td><td>$2'$</td><td>$2'$</td><td>$1 \oplus 1' \oplus 2'$</td><td>$1'' \oplus 1''' \oplus 2$</td></tr><tr><td>$2'$</td><td>$2'$</td><td>$2$</td><td>$2$</td><td>$1'' \oplus 1''' \oplus 2$</td><td>$1 \oplus 1' \oplus 2'$</td></tr></tbody></table>

those of the doublets are defined as follows:

$$ 
\begin{gathered}
\begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \otimes \begin{pmatrix} y_1 \\ y_2 \end{pmatrix} = (x_1 y_2 - x_2 y_1) \oplus (x_1 y_2 + x_2 y_1) \oplus \begin{pmatrix} x_1 y_1 \\ -x_2 y_2 \end{pmatrix}, \\
\mathbf{2} \otimes \mathbf{2} = \mathbf{1} \oplus \mathbf{1}' \oplus \mathbf{2}', \\
\begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \otimes \begin{pmatrix} y_1 \\ y_2 \end{pmatrix} = (x_1 y_2 + x_2 y_1) \oplus (x_1 y_2 - x_2 y_1) \oplus \begin{pmatrix} x_2 y_2 \\ x_1 y_1 \end{pmatrix}, \qquad (7.28) \\
\mathbf{2}' \otimes \mathbf{2}' = \mathbf{1} \oplus \mathbf{1}' \oplus \mathbf{2}', \\
\begin{pmatrix} x_1 \\ x_2 \end{pmatrix} \otimes \begin{pmatrix} y_1 \\ y_2 \end{pmatrix} = (x_1 y_1 - x_2 y_2) \oplus (x_1 y_1 + x_2 y_2) \oplus \begin{pmatrix} x_2 y_1 \\ x_1 y_2 \end{pmatrix}. \\
\mathbf{2} \otimes \mathbf{2}' = \mathbf{1}'' \oplus \mathbf{1}''' \oplus \mathbf{2}
\end{gathered}
$$

The complex conjugation of **2**, i.e. $\mathbf{2}^*$, is obtained by $i\tau_2\mathbf{2}$, where $\tau_2$ is the second Pauli matrix.

We demonstrate how the mass hierarchy and the mass degeneracy can be realized in the presence of $Q_6$ with a simple example. The $Q_6$ representations are assigned as

$$ l_L \sim \mathbf{1}, \quad \nu_{R,1} \sim \mathbf{1}''', \quad \nu_{R,D} = (\nu_{R,2} \ \nu_{R,3}) \sim \mathbf{2}', \quad \Phi \sim \mathbf{1}, \quad D \sim \mathbf{2}, $$

so that a mass term of $\overline{\nu_{R,1}^c} \nu_{R,1}$ is not allowed, and $\overline{\nu_{R,D}^c} \nu_{R,D}$ results in degenerate masses. The left-handed lepton doublet, $l_L$, and the SM Higgs doublet, $\Phi$, are assumed to be **1**; therefore, the charged lepton sector is the same as that of the SM. In order to spontaneously break the $Q_6$ symmetry, we introduce a $Q_6$ doublet scalar $D$ which is SM gauge singlet, and we also take into account non-renormalizable effective operators with $D$. Then, the Dirac and Majorana mass terms are given by

$$ -\mathcal{L}_D = \frac{f_\alpha}{\Lambda^2} \overline{l_{L,\alpha}} \nu_{R,D} \tilde{\Phi} D D + \frac{\tilde{f}_\alpha}{\Lambda^2} \overline{l_{L,\alpha}} \nu_{R,D} \tilde{\Phi} D D^* + \frac{F_{\alpha 1}}{\Lambda^3} \overline{l_{L,\alpha}} \nu_{R,1} \tilde{\Phi} D D D + \dots + \text{h.c.}, \qquad (7.29) $$

$$ -\mathcal{L}_M = \frac{1}{2} \left( M_d \overline{\nu_{R,D}^c} \nu_{R,D} + \frac{M_a}{\Lambda} \overline{\nu_{R,1}^c} \nu_{R,D} D + 2 \frac{M_b}{\Lambda^2} \overline{\nu_{R,D}^c} \nu_{R,D} D D + \frac{M_c}{\Lambda^2} \overline{\nu_{R,1}^c} \nu_{R,1} D D + \dots + \text{h.c.} \right), \qquad (7.30) $$
respectively, where $\alpha = e, \mu, \tau$, $\Lambda$ stands for the $Q_6$ symmetry breaking scale, which is supposed to be very high, and $\cdots$ represents higher order terms. Note that we have omitted several terms whose contributions can be absorbed into other terms and implicitly assumed a mechanism which provides $M_{a,b,c,d} = \mathcal{O}(10)$ GeV, e.g., spontaneous breaking of a lepton number symmetry at the GeV scale. After $D$ (and $\Phi$) acquire vacuum expectation values, $\langle D \rangle = (d_1 d_2)^T$, the Dirac and Majorana mass matrices are obtained as

$$m_D = v\lambda^2 \begin{pmatrix} 0 & F_{e2} & F_{e3} \\ 0 & F_{\mu2} & F_{\mu3} \\ 0 & F_{\tau2} & F_{\tau3} \end{pmatrix} + v\lambda^3 \begin{pmatrix} F_{e1} & 0 & 0 \\ F_{\mu1} & 0 & 0 \\ F_{\tau1} & 0 & 0 \end{pmatrix}, \qquad (7.31)$$

$$M_M = \begin{pmatrix} 0 & 0 & 0 \\ 0 & 0 & M_d \\ 0 & M_d & 0 \end{pmatrix} + \lambda \begin{pmatrix} 0 & M_a & -M_a \\ M_a & 0 & 0 \\ -M_a & 0 & 0 \end{pmatrix} + \lambda^2 \begin{pmatrix} 2M_c & 0 & 0 \\ 0 & M_b & 0 \\ 0 & 0 & -M_b \end{pmatrix}. \qquad (7.32)$$

In the above equations, we assume $d_1 = d_2 = d$ for simplicity and define $\lambda = d/\Lambda$, $F_{\alpha 2} = -f_{\alpha} + \tilde{f}_{\alpha}$, and $F_{\alpha 3} = f_{\alpha} + \tilde{f}_{\alpha}$. Given $\lambda \ll 1$, the masses of the sterile neutrinos are found to be

$$M_1 \approx 2\lambda^2 \left| M_c + \frac{M_a^2}{M_d} \right|, \quad M_{2,3} \approx |M_d \pm \lambda^2 M_b|. \qquad (7.33)$$

As can be seen, $M_1$ is suppressed by $\lambda^2$ in comparison with $M_{2,3}$, and nearly degenerate $M_2$ and $M_3$ are realized. Moreover, very small $(m_D)_{\alpha 1}$ are obtained, which may result in a small active-sterile mixing.

Suppose $M_1 = \mathcal{O}(1)$ keV and $M_{2,3} = \mathcal{O}(10)$ GeV, then $\lambda \simeq 10^{-3.5}$ is required, leading to $|M_2 - M_3| = \mathcal{O}(10^{-7})$ GeV. The mass scale of $M_{2,3}$ is chosen to be suitable for baryogenesis via flavor oscillation [188, 398, 806]. Interestingly, $|M_2 - M_3| = \mathcal{O}(10^{-7})$ GeV coincides with the mass difference obtained in ref. [398]. Thanks to the suppression by $\lambda$, the active neutrino masses, $m_\nu = \mathcal{O}(10^{-11})$ GeV, are reproduced with $F_{\alpha 2}, F_{\alpha 3} = \mathcal{O}(10^{-1} - 1)$ even for $M_{2,3} = \mathcal{O}(10)$ GeV). Lastly, a small active-sterile mixing

$$\theta_{\alpha 1} = \frac{(m_D)_{\alpha 1}}{M_1} = \frac{v\lambda^3 F_{\alpha 1}}{M_1} \approx 10^{-5} - 10^{-4}, \qquad (7.34)$$

is successfully realized by demanding $F_{\alpha 1} = \mathcal{O}(10^{-2.5} - 10^{-1.5})$.

Last of all, let us comment on flavor mixing of the active neutrinos. $Q_6$ includes $\mu - \tau$ symmetry as subgroup, and one can exploit it to derive the observed large atmospheric and small reactor mixing angles [240].

### 7.3.3 $A_4$ symmetry (author: A. Merle)

An alternative route to pursue with flavor symmetries is to use them to explain only the correct active and active-sterile mixing patterns, while using a different mechanism for the mass suppression. This means that, contrary to the approach taken in section 7.3.2, one could take an existing mass suppression mechanism and supplement it by a flavor symmetry. Here we focus on an example present in the literature [174, 807] which uses the FN mechanism [202], along with an auxiliary $A_4$ discrete symmetry. The FN mechanism has been shown in section 7.2.2 to be able to generate the desired mass patterns. The whole course from the group theory of $A_4$ over developing concrete models down to experimental tests is
outlined for example in ref. [219]. Here, we will simply focus on how to use this symmetry as an auxiliary tool to tweak a setting with a “pre-existing” mass suppression mechanism in such a way that a phenomenologically acceptable model is obtained.

The symmetry group $A_4$ is a subgroup of SU(3) [808], and it consists of twelve different elements and it admits four irreducible representations: **1**, **1'**, **1''**, and **3**. The group has two generators $S$ and $T$ which obey the relations $S^2 = T^3 = (ST)^3 = 1$ [219]. The model presented here already contains a suitable mass pattern generated purely by the FN mechanism, with one light sterile neutrino $\nu_{R,1}$ at the keV scale (with FN charge $F_1$), while two sterile (right-handed) neutrinos $\nu_{R,2/3}$ with FN charges $F_{2,3}$ are considerably heavier (GeV or higher). This model needs two Higgs doublets $H_{u,d}$, too, as well as seven flavons $\phi$, $\phi'$, $\phi''$, $\xi$, $\xi'$, $\xi''$, and $\Theta$. Among these fields, $\xi$ obtains a VEV $u = \langle\xi\rangle$ which generates the lightest sterile neutrino mass according to $M_1 \propto u\lambda^{2F_1}$, where $u \simeq 10^{12}$ GeV and $\lambda \sim 0.22$, such that the choice $F_1 = 9$ seems appropriate to get a keV scale mass for $\nu_{R,1}$.

The most general Lagrangian at leading order in the family symmetry breaking scale, using the charge assignment of table 14, is given by

$$ \mathcal{L}_Y = -\frac{y_e}{\Lambda} \lambda^3 \overline{e_R} (\varphi H_d L)_1 - \frac{y_\mu}{\Lambda} \lambda \overline{\mu_R} (\varphi H_d L)_{1'} - \frac{y_\tau}{\Lambda} \overline{\tau_R} (\varphi H_d L)_{1''} \\ -\frac{y_1}{\Lambda} \lambda^{F_1} \overline{\nu_{R,1}} (\varphi H_u L)_1 - \frac{y_2}{\Lambda} \lambda^{F_2} \overline{\nu_{R,2}} (\varphi' H_u L)_{1''} - \frac{y_3}{\Lambda} \lambda^{F_3} \overline{\nu_{R,3}} (\varphi'' H_u L)_1 \\ -\frac{1}{2} \left[ w_1^* \lambda^{2F_1} \xi^* \overline{(\nu_{R,1})^c} \nu_{R,1} - w_2^* \lambda^{2F_2} \xi'^* \overline{(\nu_{R,2})^c} \nu_{R,2} - w_3^* \lambda^{2F_3} \xi''^* \overline{(\nu_{R,3})^c} \nu_{R,3} \right] + \text{h.c.}, \quad (7.35) $$

where the notation $(...)_x$ indicates the irreducible $A_4$ representation $x$ to which the respective terms belong to and $\Lambda$ denotes a high scale. With a VEV $u = \langle\xi\rangle$, the first term in the last line indeed leads to a keV neutrino mass of

$$ M_1 = w_1 u \lambda^{2F_1}. \quad (7.36) $$

Looking again at table 14, one can see that the field $\nu_{R,1}$ is, in fact, a total singlet under all symmetries but the U(1)$_{FN}$. Thus, as anticipated, its mass is exclusively determined by the FN symmetry, except for the rather indirect influence of the VEV $u$ of $\xi$.

What is then the actual use of the $A_4$ symmetry? This can be seen by considering the active neutrino mixing. Illustrating the case of normal mass ordering, one can use the alignments $\langle\phi'\rangle = (v'_{\phi}, v'_{\phi'}, v'_{\phi''})$ and $\langle\phi''\rangle = (0, v''_{\phi}, -v''_{\phi''})$ to obtain a phenomenologically viable $5 \times 5$ neutrino mass matrix.<fnref n="42" /> Calculating the corresponding leptonic mixing to next-to-leading order, one obtains a mixing matrix given by

$$ U_{\nu}^{(\text{NO})} \approx \begin{pmatrix} \frac{2}{\sqrt{6}} & \frac{1}{\sqrt{3}} & 0 & 0 & 0 \\ -\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{3}} & -\frac{1}{\sqrt{2}} & 0 & 0 \\ -\frac{1}{\sqrt{6}} & \frac{1}{\sqrt{3}} & \frac{1}{\sqrt{2}} & 0 & 0 \\ 0 & 0 & 0 & 1 & 0 \\ 0 & 0 & 0 & 0 & 1 \end{pmatrix} + \begin{pmatrix} 0 & 0 & 0 & \epsilon_1 & 0 \\ 0 & 0 & 0 & \epsilon_1 & -\epsilon_2 \\ 0 & 0 & 0 & \epsilon_1 & \epsilon_2 \\ 0 & -\sqrt{3}\epsilon_1 & 0 & 0 & 0 \\ 0 & 0 & -\sqrt{2}\epsilon_2 & 0 & 0 \end{pmatrix} + \mathcal{O}(\epsilon_i^2), \quad (7.37) $$

where $\epsilon_{1,2}$ are small parameters arising only at higher order which features the other two FN charges $F_{2,3}$. It is thus possible to see the actual effect of the $A_4$ symmetry: it leads to

<footnote n="42">⁴²One can show that the keV neutrino practically decouples, in the sense that its presence does hardly affect the masses or mixings of the other neutrino mass eigenstates, so that it is sufficient to take into account $\nu_{2,3R}$.</footnote>
<table><caption>**Table 14.** Particle content and representation/charge assignments model from ref. [807]. In addition to $A_4$ and the FN symmetry, another auxiliary symmetry $\mathbb{Z}_3$ is used in order for CP violation to survive. For $SU(2)_L$ and $A_4$ the representations are reported, while for $\mathbb{Z}_3$ and $U(1)_{FN}$ the charge is given, where $\omega = e^{2\pi i/3}$.</caption><thead><tr><th>Field</th><th><i>L</i><sub>1,2,3</sub></th><th><i>e</i><sub>R</sub></th><th><i>μ</i><sub>R</sub></th><th><i>τ</i><sub>R</sub></th><th><i>ν</i><sub>R,1</sub></th><th><i>ν</i><sub>R,2</sub></th><th><i>ν</i><sub>R,3</sub></th><th><i>H</i><sub>u,d</sub></th><th><i>φ</i></th><th><i>φ</i>'</th><th><i>φ</i>''</th><th><i>ξ</i></th><th><i>ξ</i>'</th><th><i>ξ</i>''</th><th><i>Θ</i></th></tr></thead><tbody><tr><td>SU(2)<sub>L</sub></td><td><b>2</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>2</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td><td><b>1</b></td></tr><tr><td><i>A</i><sub>4</sub></td><td><b>3</b></td><td><b>1</b></td><td><b>1</b>''</td><td><b>1</b>'</td><td><b>1</b></td><td><b>1</b>'</td><td><b>1</b></td><td><b>1</b></td><td><b>3</b></td><td><b>3</b></td><td><b>3</b></td><td><b>1</b></td><td><b>1</b>'</td><td><b>1</b></td><td><b>1</b></td></tr><tr><td><i>Z</i><sub>3</sub></td><td><i>ω</i></td><td><i>ω</i><sup>2</sup></td><td><i>ω</i><sup>2</sup></td><td><i>ω</i><sup>2</sup></td><td>1</td><td>1</td><td><i>ω</i></td><td><i>ω</i><sup>2</sup></td><td><i>ω</i><sup>2</sup></td><td><i>ω</i></td><td>1</td><td>1</td><td><i>ω</i><sup>2</sup></td><td><i>ω</i></td><td>1</td></tr><tr><td>U(1)<sub>FN</sub></td><td>0</td><td>3</td><td>1</td><td>0</td><td><i>F</i><sub>1</sub></td><td><i>F</i><sub>2</sub></td><td><i>F</i><sub>3</sub></td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>0</td><td>-1</td></tr></tbody></table>

phenomenologically acceptable leptonic mixing. While the leading order terms generate a tri-bimaximal mixing matrix, which in particular features $\theta_{13} \equiv 0$ that is excluded as well as maximal $\theta_{23}$, the terms arising at next to leading order correct the mixing pattern such that a non-zero $\theta_{13}$, which can indeed arise from the sterile neutrino sector [809], and a deviation of $\theta_{23}$ from $\pi/4$ are obtained at the same time. Thus, while the smallness of the keV neutrino mass is achieved by a completely separated sector, the $A_4$ symmetry adds another important ingredient.

Let us finally comment on an aspect which relies on both ingredients. While a keV neutrino does not affect the parameters of the light neutrino sector, it mixes with the active neutrinos via the mixing angles $\theta_{\alpha 1}$ — whose values are tightly bound by the X-ray limit. This mixing is enhanced by the relatively high FN charge according to

$$ \theta_{e1} \approx \frac{y_1 v_{\varphi} v_u}{w_1 u \Lambda} \lambda^{-F_1}, \qquad (7.38) $$

with $\theta_{\mu 1, \tau 1}$ being of similar order. Thus, a too large value of $F_1$ might destroy the validity of the model. However, in practice, there is typically enough freedom to vary the couplings and VEVs involved, such that this constraint is avoided. On the other hand, if the sterile neutrino sector was known better, the link between the different sectors could be exploited to probe the model.

In this section, we could only give a glance of the aspects of the $A_4$ model presented and the benefits of combining different ingredients. To learn more about the detailed aspects, we refer the reader to the original references [174, 807] as well as to the pedagogical review [66].

## 7.4 Models based on other principles

Finally, there also exist models based on other principles which do not belong to one of the categories discussed up to now.

### 7.4.1 Extended seesaw (author: J. Heeck)

The extended seesaw (ES) [810, 811] aims to explain the smallness of one or more sterile neutrinos by the very same seesaw mechanism that is behind small active neutrino masses.<fnref n="43" /> In order for the light sterile neutrino(s) to mimic the seesaw couplings of active neutrinos, a structure in the mass matrix of neutral fermions is necessary, to be invoked by a symmetry.

<footnote n="43">⁴³While most often employed to generate sterile neutrinos at the eV scale [159], the ES is also applicable to keV sterile neutrinos, since their mass is still far below the generic seesaw scale.</footnote>
On top of the standard seesaw scenario with $n_R$ right-handed singlets $\nu_R$ with Majorana mass matrix $M_M$ and Dirac mass terms $m_D \propto \langle\Phi\rangle$ from electroweak symmetry breaking

$$ \mathcal{L} \supset -\overline{\nu_L} m_D \nu_R - \frac{1}{2} \overline{\nu_R^c} M_M \nu_R + \text{h.c.}, \qquad (7.39) $$

the ES introduces $n_S$ left-handed SM-singlet fermions $S_L$ with ad hoc $\nu_L$-like couplings

$$ \mathcal{L}_S = -\overline{S_L} m_N \nu_R + \text{h.c.} \qquad (7.40) $$

Here, $m_N$ is a mass scale in principle unrelated to electroweak symmetry breaking but typically also generated by a vacuum expectation value (see discussion of models below). The full $(3+n_R+n_S) \times (3+n_R+n_S)$ ES mass matrix for $(\nu_L^c, \nu_R, S_L^c)$ then takes the form [174, 774]

$$ M_{\text{ES}}^{3+n_R+n_S} = \begin{pmatrix} 0 & m_D & 0 \\ m_D^T & M_M & m_N^T \\ 0 & m_N & 0 \end{pmatrix}, \qquad (7.41) $$

leading, in the seesaw limit $M_M \gg m_D, m_N$, to a low-energy mass matrix for the $3+n_S$ light neutrinos suppressed by the high seesaw scale $M_M$

$$ M_{\nu}^{3+n_S} \approx - \begin{pmatrix} m_D M_M^{-1} m_D^T & m_D M_M^{-1} m_N^T \\ m_N M_M^{-1} m_D^T & m_N M_M^{-1} m_N^T \end{pmatrix}, \qquad (7.42) $$

as intended. Since the $S_L$ mimic left-handed neutrinos by construction, standard arguments [158] tell us that at most $n_R$ of the $3+n_S$ light neutrinos gain mass by this mechanism.

We will in the following consider the minimal extended seesaw [174, 774] ($n_S = 1$ and $n_R = 3$) for simplicity, which leads to one massless (active) neutrino. In the case of interest for a keV sterile neutrino, we consider $m_N \gg m_D$, allowing us to use the seesaw formula once more on $M_\nu^{3+1}$ and find the keV neutrino mass, $m_s \simeq |m_N M_M^{-1} m_N^T|$, and active-sterile mixing angle,

$$ \theta_{\text{as}} \equiv m_D M_M^{-1} m_N^T (m_N M_M^{-1} m_N^T)^{-1} = \mathcal{O}(m_D/m_N) = \mathcal{O}(\sqrt{m_\nu}/\sqrt{m_s}). \qquad (7.43) $$

From the naive scaling in the last two equations we see that small cancellations in the matrix structure $m_D M_M^{-1} m_N^T$ are necessary to obtain $m_s = \mathcal{O}(\text{keV})$ with a small enough mixing $\theta_{\text{as}}$ to satisfy X-ray bounds [774]. All in all, the minimal ES can however quite naturally accommodate a weakly coupled keV sterile neutrino without the need of unnaturally small couplings. The price to pay is the introduction of a third mass scale, $m_N \sim m_D \sqrt{m_s/m_\nu}$, somewhat above the electroweak scale, plus a mechanism/symmetry to actually generate the desired ES structure of eq. (7.41). $S_L$ must carry some (broken) quantum numbers to distinguish it from the $\nu_R$, because a full singlet would also allow for the unwanted couplings

$$ -\overline{l_L} \left( \frac{m_D^S}{v} \right) \tilde{\Phi} S_L^c - \frac{1}{2} \overline{S_L} \mu_S S_L^c + \text{h.c.} \supset -\overline{\nu_L} m_D^S S_L^c - \frac{1}{2} \overline{S_L} \mu_S S_L^c + \text{h.c.} \qquad (7.44) $$

The model-building exercise for ES is hence to forbid the terms in eq. (7.44) and only allow for those in eq. (7.40), which can be accomplished by imposing symmetries.

The most straightforward approach is to introduce a SM-singlet scalar $\phi$ that carries the same non-trivial charge as $S_L$ under some global symmetry group $\mathcal{G}$, allowing for the coupling
$\overline{S}_L\phi\nu_R$ and hence generating $m_N \propto \langle\phi\rangle$ upon symmetry breaking of $\mathcal{G}$ via the VEV of $\phi$. $\mathcal{G}$ has to be chosen such that the couplings in eq. (7.44) remain forbidden at the renormalizable level, which excludes the use of $\mathbb{Z}_2$ and $\mathbb{Z}_3$, as they would allow for the naturally large Majorana mass terms $\overline{S}_L\mu_S S_L^c$ and $\overline{S}_L\langle\bar{\phi}\rangle S_L^c$, respectively. $\mathcal{G} = \mathbb{Z}_4$ emerges as the simplest choice to realize the ES structure with global symmetries [169]; the $\mathbb{Z}_4$ invariant term $(\phi^4+\bar{\phi}^4)$ in the scalar potential breaks the accidental U(1) symmetry one encounters for $\mathcal{G} = \mathbb{Z}_{n>5}$ and thus removes a potentially problematic Goldstone boson from the spectrum.<fnref n="44" /> In ref. [774], it was further shown how this symmetry can be incorporated in the flavor symmetry group $A_4 \times \mathbb{Z}_4$ in order to additionally shed light on the lepton mixing pattern. Other choices of $\mathcal{G}$ without Goldstone bosons can be realized with non-minimal scalar sectors [810].

In a different approach, one can choose $\mathcal{G}$ to be a gauge symmetry in order to avoid potential drawbacks of discrete global symmetries such as incompatibility with quantum gravity [812] or domain walls [813]. Since non-Abelian symmetries $\mathcal{G}$ generate (typically) more than one light sterile neutrino [169, 814, 815], we focus on $\mathcal{G} = \text{U}(1)$ here. Renormalizability requires the introduction of additional chiral fermions $T$ which cancel the gauge anomaly from the chiral $S_L$ [169, 775, 814, 815]. In order for the $T$ not to spoil the ES matrix structure of eq. (7.41), $S$ and $T$ can be chosen to form an anomaly-free chiral set of U(1) charges [816], somewhat reminiscent of the hypercharge assignment in the SM, for example as [775]

$$ (S, T_1, T_2, T_3, T_4, T_5, T_6) \sim (11, -5, -6, 1, -12, 2, 9). \qquad (7.45) $$

The U(1) symmetry breaking by a scalar $\phi \sim 11$ couples $S$ to the right-handed neutrinos in the ES manner and also generates a mass for the new gauge boson $Z'$, as well as for the $T_j$ via

$$ \mathcal{L}_T = y_1 \overline{T}_1^c T_2 \phi + y_2 \overline{T}_3^c T_4 \phi + y_3 \overline{T}_5^c T_6 \bar{\phi} + \text{h.c.} \qquad (7.46) $$

The six anomaly-canceling $T_j$ thus form three massive Dirac fermions and have no impact on the ES mechanism. As emphasized in ref. [775], these Dirac fermions are stabilized by the remaining $\mathbb{Z}_{11}$ symmetry, after U(1) symmetry breaking (similar to ref. [817]), and are coupled to the SM by (kinetic) $Z-Z'$ mixing –the Higgs portal $|H|^2|\phi|^2$– and via active-sterile neutrino mixing. A consistent realization of the ES structure by means of a gauge symmetry thus automatically leads to (multi-component and self-interacting) (C)DM candidates. The keV neutrino $\nu_s \sim S$ is charged under the U(1) gauge symmetry and hence strictly speaking not sterile, but can be consistent with observational constraints. More than one light sterile neutrinos can be obtained using different chiral sets [169, 814, 815], which can also modify the phenomenology of the CDM component [775].

### 7.4.2 Dynamical mass generation and composite neutrinos (authors: D. Robinson, Y. Tsai)

Most keV neutrino model-building frameworks require Majorana neutrinos. Here, in contrast, we outline a framework that dynamically produces both sub-eV active and keV sterile Dirac neutrinos. The central idea [754, 818] is that the right-handed neutrinos are composite states of a hidden confining sector [819–824], while the keV steriles are elementary spectators that gain their masses in a similar fashion to the quarks in extended Technicolor theories. The Dirac-type mass structure of the active and sterile states typically fixes their mixing angle $\theta_{\alpha I} \sim m_\alpha/m_I$. This feature is particularly appealing in light of the recent, putative

<footnote n="44">⁴⁴The Goldstone boson $J \propto \text{Im}(\phi)$ would enable the fast decay $\Gamma(\nu_s \to \nu_\alpha J) \propto \theta_{\alpha s}^2 m_s^3 / \langle\phi\rangle^2$, unless the U(1) breaking scale is much higher than expected from $m_N \sim \sqrt{m_s/m_\nu m_D}$, say $\langle\phi\rangle > 10^9$ GeV.</footnote>
3.55 keV X-ray line result [190, 191]. Should the source turn out to be sterile neutrino DM with $m_I \simeq 7.1$ keV, then the observed flux implies a mixing angle $\theta_{\alpha I} \simeq 4 \times 10^{-6}$: this is comparable to the active-sterile mass ratio $m_\alpha/m_I \sim 7 \times 10^{-6}$ for a typical active neutrino mass of $m_\alpha \sim 0.05$ eV.

In addition to the SM field content, $\psi_{SM}$, suppose that there exists a hidden sector of chiral fermions, $\chi$ and $\xi$ — there are generically multiple species of each — charged under a hidden confining group (hidden flavor symmetry) $G_c$ ($G_F$), such that $\psi_{SM} \sim G_{SM} \times G_F$, $\chi \sim G_c \times G_F$, and $\xi \sim G_F$. We consider this to be a low-energy effective field theory below some UV completion scale, $M$, so that the SM and hidden sectors interact via $M$-scale irrelevant operators. At a scale $\Lambda \ll M$, $G_c$ becomes strongly coupled, inducing $\chi$ confinement into hidden bound states. The $\xi$ are spectators, required to cancel the $G_F$ anomalies.

In order to avoid flavor constraints, we necessarily need $M \gg v$. It is therefore convenient to define two parameters $\varepsilon \equiv \Lambda/M \ll 1$ and $\vartheta \equiv v/M \ll 1$. With appropriate choices for the hidden sector, one may ensure: (i) Confinement of the $\chi$ induces the breaking $G_F \to U(1)_F$; (ii) Anomaly matching between the free and confined phases requires that there are precisely three massless chiral bound states, $\nu_R$, all with the same $F$ charge; (iii) $F$ is a linear combination of $B-L$ and hypercharge such that $U(1)_{EM} \times U(1)_{B-L}$ survives electroweak symmetry breaking (EWSB); (vi) The $\nu_R$ have appropriate $F$ charge to be right-handed neutrinos, i.e. they form the $\overline{l}_L \Phi^\dagger \nu_R$ Yukawas; (v) Along with the three $\nu_R$, there are fermionic bound states (spectators) $N_L^c$ and $N_R$ ($\xi_L^c$ and $\xi_R$) which furnish Dirac multiplets, and have the correct $F$ charge to be sterile neutrinos; (vi) The composite $\nu_R$ and $N_{R,L}$ contain three $\chi$'s, i.e. $\nu_R \sim \chi^3$ etc., and the symmetry breaking condensate has two $\chi$'s, i.e. $\langle \chi^2 \rangle \neq 0$. Examples of toy models that incorporate these features are included in ref. [818].

After $\chi$-confinement, suppressed Yukawa couplings between the active lepton doublet and composite neutrinos are generated via dimensional transmutation, viz.

$$ \frac{\overline{l}_L \Phi^\dagger \chi^3}{M^3} \to \frac{\varepsilon^3}{(4\pi)^2} \overline{l}_L \Phi^\dagger \nu_R \text{ or } \frac{\varepsilon^3}{(4\pi)^2} \overline{l}_L \Phi^\dagger N_R, \quad \text{and} \quad \frac{\overline{l}_L \Phi^\dagger \chi^2 \xi}{M^3} \to \frac{\varepsilon^3}{(4\pi)^2} \overline{l}_L \Phi^\dagger \xi_R. \quad (7.47) $$

The spectators, however, obtain Dirac masses through irrelevant couplings to the condensate VEV,

$$ \frac{\xi \langle \chi^2 \rangle \xi}{M^2} \to \frac{\Lambda \varepsilon^2}{(4\pi)^2} \overline{\xi_L} \xi_R. \quad (7.48) $$

One may require that the $G_F$ structure does not admit Dirac mass cross-terms of composites with $\xi$'s. By analogy with large-$N_c$ QCD combinatorics the three-$\chi$ massive bound states $N_{L,R}$ typically have a Dirac mass $M_N \sim 3\Lambda$. After EWSB, one then obtains a mass term of block-matrix form [754, 818],

$$ \frac{\Lambda}{(4\pi)^2} \left( \overline{\nu_L} \overline{\xi_L} \overline{N_L} \right) \begin{pmatrix} \vartheta \varepsilon^2 & \vartheta \varepsilon^2 & \vartheta \varepsilon^2 \\ 0 & \varepsilon^2 & 0 \\ 0 & 0 & 3(4\pi)^2 \end{pmatrix} \begin{pmatrix} \nu_R \\ \xi_R \\ N_R \end{pmatrix}. \quad (7.49) $$

Up to $O(1)$ factors, the neutrino masses and mixing angles are now fixed by the two dynamical scales $M$ and $\Lambda$ via eq. (7.49). The spectrum corresponding to eq. (7.49) is

$$ m_\alpha \sim v\varepsilon^3/(16\pi^2), \quad m_{\xi I} \sim \Lambda\varepsilon^2/(16\pi^2), \quad m_{NI} \sim 3\Lambda. \quad (7.50) $$
Hence there are Dirac neutrinos $\nu_{L,R}^{\alpha}$ ($\xi_{L,R}^{I}$) whose masses are dynamically $\epsilon^{3}$-($\epsilon^{2}$-)suppressed compared to the confinement scale. The corresponding left-handed neutrino mass basis

$$ \begin{pmatrix} \nu_L^\alpha \\ \xi_L^I \\ N_L^I \end{pmatrix} \sim \begin{pmatrix} 1 & \vartheta & \frac{\vartheta \epsilon^2}{48 \pi^2} \\ \vartheta & 1 & 0 \\ \frac{\vartheta \epsilon^2}{48 \pi^2} & 0 & 1 \end{pmatrix} \begin{pmatrix} \nu_L \\ \xi_L \\ N_L \end{pmatrix}, \qquad (7.51) $$

from which one sees that the active-keV sterile mixing angle $\theta_{\alpha I}$ (i.e. the angle spanned by $\nu_L^\alpha$ and $\xi_L^I$) is given by $\theta_{\alpha I} \sim \vartheta \equiv v/M$. Constraining the active neutrino mass $m_\alpha < 0.05$ eV immediately requires $\epsilon \lesssim 5 \times 10^{-4}$. For the benchmark choices $\epsilon = 3 \times 10^{-4}$ and $\Lambda = 15$ TeV — i.e. $M \sim 5 \times 10^4$ TeV, certainly flavor safe — then $m_\alpha \sim 0.05$ eV, $m_{\xi I} \sim 5$ keV and $\sin^2(2\theta_{\alpha I}) \sim 5 \times 10^{-11}$.

With a keV scale mass and a mixing angle of this order, the elementary spectators $\xi$ serve as potential WDM candidates. We emphasize that in this composite neutrino framework, the two dynamical scales $\Lambda$ and $M$ — or, equivalently, $\Lambda$ and $\epsilon$ — fix the scales of the spectrum and couplings, and hence also the approximate cosmological history of $\xi$. For example, the $\xi$, $\chi$, SM sectors are thermalized via M-scale irrelevant interactions, with decoupling temperature $T_{\text{dec}} \sim (\Lambda^4/\epsilon^4 M_{\text{pl}})^{1/3}$ [754]. Remarkably, $T_{\text{dec}} \sim \Lambda$ for the benchmark $\Lambda$ and $\epsilon$. The keV $\xi$'s are therefore thermally produced while ultrarelativistic, resulting in an overclosed Universe. Significant entropy production is then required to obtain a DM-like relic abundance.

Intriguingly, since $\Lambda \sim T_{\text{dec}}$, one possible source of such entropy could be a strongly supercooled $\chi$-confining phase transition just after $\xi$ decoupling, but just before hidden bound state-SM decoupling. Such supercooling deposits significant entropy into the hidden bound state and SM sectors, diluting the decoupled $\xi$'s [754]. However, this option generally requires the ratio of the confinement and phase transition temperatures to be $\sim 6$, which seems implausibly large.

An alternative is the late, out-of-equilibrium decay of a heavy state. Conveniently, at least one heavy composite neutrino species, $N_h(\equiv N_{L,R})$, is generically present in this framework. If the decay of $N_h$ proceeds dominantly through $N_h \to \nu_L^\alpha h$, the $\epsilon^3$-suppressed coupling (7.47) produces a sufficiently small decay rate $\Gamma_{N_h} \sim 9\Lambda\epsilon^6/4096\pi^5$ — the benchmark lifetime is $\lesssim 10^{-2}$ s — such that sizable entropy production occurs right before the BBN epoch. The resulting entropy production factor, $\gamma$, and the $\xi$ relic density are respectively

$$ \gamma \sim m_{N_h} Y_{N_h} / \Gamma_{N_h}^{1/2} M_{\text{pl}}^{1/2}, \qquad \Omega_{\xi}/\Omega_{\text{DM}} = m_{\xi}(Y_{\xi}/\gamma) s_0 \rho_{\text{DM}}^{-1}. \qquad (7.52) $$

Dominance of the $N_h \to \nu_L^\alpha h$ decay channel can happen, e.g., if the composite neutrino species have nearly democratic couplings and suppressed hidden-pion couplings. A more detailed discussion can be found in ref. [818]. The yield $Y_{N_h}$ is then determined by $\Lambda$ and $\epsilon$ up to $O(1)$ factors, just as were its mass and decay rate. The $\xi$ relic density is then highly predictable: at the benchmark values, one finds that $\Omega_{\xi} \lesssim \Omega_{\text{DM}}$.

Being a Dirac fermion, note that $\xi$ has four degrees of freedom per species, making $\xi$ WDM slightly cooler compared to typical sterile Majorana neutrino DM. This permits to evade structure formation bounds slightly more easily, including the constraints from Lyman-$\alpha$ bounds [328, 331, 553] and N-body simulations of subhalo formation [486], see ref. [818] for details.
### 7.4.3 3-3-1-models (authors: A.G. Dias, N. Anh Ky, C.A. de S. Pires, P.S. Rodrigues da Silva, N. Thi Hong Van)

The 3-3-1 models [825–828] are distinct SM gauge extensions which can naturally contain RH neutrinos at keV scale [829–832]. These models belong to a class of constructions where the SM gauge symmetry SU(2)$_L \times U(1)_Y$ is extended to SU(3)$_L \times U(1)_X$ in such a way that the cancellation of the anomalies involves all three families, offering an explanation for the long standing family number problem [825, 833–835]. As it happens, RH neutrinos are predicted in 3-3-1 models by the reason that they form, together with the SM leptons, representations of SU(3)$_L$. We will take here one version of the models in which the RH neutrinos do not carry lepton number from the beginning and, therefore, they will not be related to active neutrinos by mean of charge conjugation. The ordinary leptons and the RH neutrino are then arranged in left-handed triplets, $L_\alpha = [\nu_{L,\alpha}, e_{L,\alpha}, N_{R,\alpha}^c]_T^T \sim (3, -1/3)$, and RH singlets $e_{R,\alpha} \sim (1, -1)$, $\alpha = e, \mu, \tau$, with the numbers in parentheses being the $(SU(3)_L, U(1)_X)$ quantum numbers.

To generate mass consistently for all fermions in this model, the following three scalar triplets are considered: $\chi = [\chi^0, \chi^-, \chi'^0]^T \sim (3, -1/3)$, $\eta = [\eta^0, \eta^-, \eta'^0]^T \sim (3, -1/3)$, and $\rho = [\rho^+, \rho^0, \rho'^+]^T \sim (3, 2/3)$. The symmetry breaking pattern is assumed as $SU(3)_L \times U(1)_X/SU(2)_L \times U(1)_Y$, occurring through the VEV $\langle \chi \rangle = [0, 0, v_{\chi'}]^T$ with $v_{\chi'} \sim \mathcal{O}(1)$ TeV, and $SU(2)_L \times U(1)_Y/U(1)_{em}$ occurring through the VEVs $\langle \eta \rangle = [v_\eta, 0, 0]^T$, $\langle \rho \rangle = [0, v_\rho, 0]^T$, with the scales related according to $v_\eta^2 + v_\rho^2 = v^2$ ($v = 174$ GeV). The scale $v_{\chi'}$ gives the main contribution to the mass of the new gauge bosons which are two electrically charged bosons, $V^\pm$, two neutral complex gauge bosons $U^0$ and $U^{0*}$, and a real neutral one, the $Z'$. Some of them, $V^\pm$ and $U^0$, carry lepton number, connecting the charged leptons to the RH neutrinos. Thus, if produced at the LHC, the decays of those new gauge bosons might imprint distinct signatures involving RH neutrinos plus SM charged leptons, for example.

Majorana mass terms for six neutrinos are obtained from dimension-five effective operators involving the leptons and scalar triplets [829–831]. This can be seen by considering a simpler case than the one analyzed in refs. [829, 830], just assuming a parity symmetry defined by $P = (-1)^{3(B-L)+2s}$, where the following fields are odd ($N_R, \rho'^+, \eta'^0, \chi^0, \chi^-, V, U) \to -1$ [836, 837].$^{45}$ Such a symmetry forbids the $\nu_L - N_R^c$ mixing which could happen through the effective operator $(\overline{L^c}\chi^*) (\eta^\dagger L)$. In this scenario the RH neutrinos are completely sterile with respect to $W^\pm$ and $Z^0$. Therefore, the effective Lagrangian containing the main operators leading to neutrino masses is

$$ \mathcal{L} \supset (\overline{L^c} \eta^*) \frac{f}{\Lambda} (\eta^\dagger L) + (\overline{L^c} \chi^*) \frac{h}{\Lambda} (\chi^\dagger L) + h.c. \quad (7.53) $$

where $f \equiv f_{\alpha\beta}$ and $h \equiv h_{\alpha\beta}$ are taken as arbitrary symmetric matrices, and $\Lambda$ as a very high energy scale. Majorana mass terms for the active and RH sterile neutrinos are generated with the VEVs $\langle \eta \rangle$ and $\langle \chi' \rangle$, leading to the mass matrices

$$ M_L = 0.6 f \left( \frac{v_\eta}{25 \, \text{GeV}} \right)^2 \left( \frac{10^{12} \, \text{GeV}}{\Lambda} \right) \, \text{eV}, \quad M_M = 10 h \left( \frac{v_{\chi'}}{3.2 \, \text{TeV}} \right)^2 \left( \frac{10^{12} \, \text{GeV}}{\Lambda} \right) \, \text{keV}. \quad (7.54) $$

Assuming $f_{\alpha\beta}, h_{\alpha\beta}$ of $\mathcal{O}(1)$ and $\Lambda \approx 10^{12}$ GeV, with the symmetry breaking scales $v_{\chi'}, v_\eta$ at TeV and tens of GeV, respectively, masses are generated at the keV scale for the RH sterile neutrinos and at the sub-eV scale for the active neutrinos. The eigenstate of the smaller eigenvalue of $M_M$ in eq. (7.54) corresponds to lightest RH neutrino and is stable,

<footnote>$^{45}$The new quarks predicted by the model are all odd under this symmetry, too.</footnote>
due the parity symmetry, as required for a particle acting as WDM [829]. Another result in this scenario is that the next-to-lightest RH neutrino, $N_{R,2}^c$, obtained from diagonalization of $M_M$ in eq. (7.54), decays into the lightest one plus a photon, $N_{R,2}^c \rightarrow N_{R,1}^c + \gamma$, through a one-loop process involving the $V^\pm$. This process might be relevant for explaining astrophysical phenomena involving X-ray lines, as for example the recent observation of the 3.5 keV line [190, 191].

A realization of how the effective operators in eq. (7.53) can be induced at low energies was done in ref. [831], assuming the existence of a heavy scalar sextet $S \sim (6, -2/3)$. In this case Majorana mass terms for neutrinos are generated through the Yukawa interaction [830, 832],

$$ \mathcal{L}_Y \supset \bar{L}^c y S^* L + \text{h.c.}, \qquad (7.55) $$

where $y \equiv y_{\alpha\beta}$, with the VEV of the sextet components $\langle S_{11} \rangle \equiv \langle \Delta^0 \rangle = v_{\Delta^0}$ and $\langle S_{33} \rangle \equiv \langle \sigma^0 \rangle = v_{\sigma^0}$. This leads to the mass matrices $[M_L]_{\alpha\beta} = y_{\alpha\beta}v_{\Delta^0}$ and $[M_R]_{\alpha\beta} = y_{\alpha\beta}v_{\sigma^0}$ for the active and RH neutrinos, respectively. The small values of $v_{\Delta^0}$ and $v_{\sigma^0}$ are due to a seesaw mechanism involving the sextet$^{46}$ and an explicit lepton number violation. Such violation occurs mainly through the terms in the potential $V \supset M_1\eta^T S^\dagger\eta + M_2\chi^T S^\dagger\chi + \text{h.c.}$, where it is assumed that $M_1 \approx M_2 \approx \Lambda$ are very high energy scales in which lepton number is violated. Thus, considering this and the fact that the mass scale of the sextet is also of order of $\Lambda \approx 10^{12}$ GeV, minimization of the potential leads to constraint equations furnishing small values for the VEVs, $v_{\Delta^0} \approx v_\eta^2/M_1$ and $v_{\sigma^0} \approx v_\eta^2/M_2$, so that $[M_L]_{\alpha\beta} \approx y_{\alpha\beta}v_\eta^2/\Lambda$ and $[M_R]_{\alpha\beta} \approx y_{\alpha\beta}v_{\chi'}^2/\Lambda$. These mass matrices have the same suppression factors as the ones in eq. (7.54). Besides, the mechanism predicts a mass relation for these two types of neutrinos, as a result of the fact that their masses arise from the same coupling $g_{\alpha\beta}$ in eq. (7.55). It can be seen that the RH/active neutrino mass ratio is $m_{N_{iR}}/m_{\nu_{iL}} = v_{\chi'}^2/v_\eta^2$ [830, 832], so that for $v_{\chi'} \approx 1$ TeV, $v_\eta \approx 10$ GeV masses at the keV scale for RH neutrinos are predicted, taking into account that the active neutrinos have masses of order $m_{\nu_{iL}} \approx 0.1$ eV [831]. The use of a scalar sextet was also developed in other ways to implement the seesaw mechanisms in 3-3-1 models [839–841].

Another model extending the SM lepton sector only by neutrino fields transforming nontrivially under a non-Abelian gauge group is the $SU(3)_L \times SU(3)_R \times U(1)_X$ model [842]. It has the essence of the left-right models [843, 844], plus the peculiarity that the number of families is restricted by gauge anomaly cancellation and by the embedding of the 3-3-1 models of refs. [827, 828]. One of the main features of the $SU(3)_L \times SU(3)_R \times U(1)_X$ model is that its rich neutrino content allows to address, simultaneously, problems related to neutrino oscillation, WDM, and baryogenesis through leptogenesis [842]. In its particle spectrum the model might have, in addition to the light active neutrinos, three neutrinos at keV mass scale, and six heavy ones having masses around $10^{11}$ GeV. Such neutrinos result from the leptonic field content $\Psi_{L,\alpha} = [\nu_{L,\alpha}, e_{L,\alpha}, N_{L,\alpha}]^T \sim (3, 1, -1/3)$ and $\Psi_{R,\alpha} = [\nu_{R,\alpha}, e_{R,\alpha}, N_{R,\alpha}]^T \sim (1, 3, -1/3)$, with the numbers in parenthesis being the $(SU(3)_L, SU(3)_R, U(1)_X)$ quantum numbers, and $N_\alpha$ being new neutral fermions.

Two sets of scalar triplets, $\eta_L, \chi_L \sim (3, 1, -1/3)$ and $\eta_R, \chi_R \sim (1, 3, -1/3)$, break the gauge symmetries and participate in the neutrinos mass generation mechanism. This sets up by means of the following effective operators [842], when the scalar fields get VEV as $\langle \eta_{L,R} \rangle = [v_{\eta_{L,R}}, 0, 0]^T$ and $\langle \chi_{L,R} \rangle = [0, 0, v_{\chi_{L,R}}]^T$,

$$ \mathcal{L} \supset (\bar{\Psi}_L \chi_L) \frac{g_X^D}{\Lambda_D} (\chi_R^\dagger \Psi_R) + (\bar{\Psi}_L^c \chi_L^*) \frac{g_X^M}{\Lambda_M} (\chi_L^\dagger \Psi_L) + (\bar{\Psi}_R^c \chi_R^*) \frac{g_X^M}{\Lambda_M} (\chi_R^\dagger \Psi_R) + (\chi \rightarrow \eta), \quad (7.56) $$

<footnote>$^{46}$This mechanism resembles the known type II seesaw mechanism [130, 197, 200, 838].</footnote>
where a $\mathbb{Z}_2$ symmetry, $\chi_{L,R} \rightarrow -\chi_{L,R}$, is assumed for simplicity. The sterile neutrinos within this model can be fully stabilized with the previous parity symmetry. The suppression scales in eq. (7.56) are such that $\Lambda_M \approx 10^{19}$ GeV, meaning that lepton number is broken by gravitational interactions at the Planck scale, and $\Lambda_D \approx 10^{15}$ GeV, which could be related to a grand unification scale. The breaking of $SU(3)_L \times SU(3)_R \times U(1)_X/SU(3)_L \times U(1)_N$ is taken to occur with the VEVs around this last scale, $v_{\eta_R} \simeq v_{\chi_R} \simeq \Lambda_D$, leaving at low energy an effective 3-3-1 model plus very heavy singlets [842]. Given that $\Lambda_M \gg \Lambda_D$, it follows from eq. (7.56) that choosing $v_{\eta_L} \approx 10$ GeV, $v_{\chi_L} \approx 1$ TeV, the mass matrix of the active neutrinos is $M_{\nu_L} \approx g_{\eta}^D (g_{\eta}^M)^{-1} (g_{\eta}^D)^T$ eV. There is also a mass matrix $M_{N_L} \approx 10 g_{\chi}^D (g_{\chi}^M)^{-1} (g_{\chi}^D)^T$ keV for three sterile neutrinos, with the lightest one being a WDM candidate. Additionally, there are six very heavy neutrinos following from the mass matrix $M_M \approx 10^{11} \text{diag}(g_M^M, g_{\chi}^M)$ GeV, which can act in the baryogenesis through leptogenesis for explaining the Universe matter-antimatter asymmetry [357].

As a final note, it is worth to point out that there are also more complete GUT settings where light sterile neutrinos can exist and be well-motivated, see the works on $E_6$ [845] as prominent example.

### 7.4.4 Anomalous Majorana neutrino masses from torsionful quantum gravity
(authors: N. Mavromatos, A. Pilaftsis)

A potentially interesting radiative mechanism for generating gauge-invariant fermion masses at three loops has been studied in [846]. It uses global anomalies triggered by the possible existence of scalar or pseudoscalar fields in $U(1)$ gauge theories and by heavy fermions $F$ whose masses may not result from spontaneous symmetry breaking. In [847], whose contents are reviewed here, this mass-generating mechanism has been “geometrized” by applying it to appropriate field theories of gravity with torsion [848–851], to create low-scale fermion masses by pure quantum gravity effects. Here, the role of the $U(1)$ gauge field strength tensor $F_{\mu\nu}$ is played by the Riemann curvature tensor $R_{\mu\nu\rho\sigma}$, and hence the role of the gauge fields by the gravitons. Such a gravitationally-assisted mass mechanism could straightforwardly be applied to fermions without SM quantum charges, such as Majorana right-handed neutrinos. Our goal is to review briefly this mechanism and provide reliable estimates of the gravitationally induced right-handed Majorana mass $M_M$. As we shall see, masses in the keV region can be naturally provided. Although quantum gravitational interactions are non-renormalizable, nevertheless there are aspects of the theory that can be exact, in a path integral sense, and these are related to some aspects of torsionful manifolds. Torsion appears as a non-propagating 3-form in a quantum gravity path integral and as such it can be integrated out exactly [848–851].

To start, consider Dirac QED fermions in a torsionful space-time. The Dirac action reads: $S_{\psi} = \frac{i}{2} \int d^4 x \sqrt{-g} (\bar{\psi} \gamma^{\mu} \overline{D}_{\mu} \psi - (\overline{D}_{\mu} \psi) \gamma^{\mu} \psi)$, where $\overline{D}_{\mu} = \overline{\nabla}_{\mu} - ieA_{\mu}$, with $e$ being the electron charge and $A_{\mu}$ the photon field. The overline above the covariant derivative, i.e. $\overline{\nabla}_{\mu}$, denotes the presence of torsion, which is introduced through the torsionful spin connection $\overline{\omega}_{ab\mu}$. The presence of torsion in the covariant derivative in the Dirac-like action leads, apart from the standard terms in manifolds without torsion, to an additional term involving the axial current $J_5^{\mu} \equiv \bar{\psi} \gamma^{\mu} \gamma^5 \psi$: $S_{\psi} \ni -\frac{3}{4} \int d^4 x \sqrt{-g} S_{\mu} \bar{\psi} \gamma^{\mu} \gamma^5 \psi = -\frac{3}{4} \int S \wedge *J^5$, where $S = *T$ is the dual of the torsion tensor $T = d e^a + \bar{\omega}^a \wedge e^b$, $S_d = \frac{1}{3!} \epsilon^{abc} T_{abc}$. It is an axial pseudovector field, one of the three types of structures appearing in the irreducible decomposition of $T$ [849, 852]. The gravitational part of the action can be written as $S_G = \frac{1}{2\kappa^2} \int d^4 x \sqrt{-g} (R + \hat{\Delta}) + \frac{3}{4\kappa^2} \int S \wedge *S$, where $R$ is the scalar curvature without torsion and
$\hat{\Delta}$ contains quadratic forms of irreducible parts of the torsion tensor other than $\mathbf{S}$. In a quantum gravity setting, where one integrates over all fields, the torsion terms appear as non-propagating fields and hence can be integrated out exactly. The authors of [852] have observed though that the classical equations of motion identify the axial-pseudovector torsion field $S_\mu$ with the axial current. It follows that $\mathbf{d}^*\mathbf{S} = 0$, leading to a conserved “torsion charge” $Q = \int \mathbf{d}^*\mathbf{S}$. To maintain this conservation in the quantum theory, they postulated $\mathbf{d}^*\mathbf{S} = 0$ at the quantum level — which can be achieved by the addition of judicious counter terms. This constraint, in a path-integral formulation of quantum gravity, is then implemented via a $\delta$-function, $\delta(\mathbf{d}^*\mathbf{S})$, and the latter via the well-known trick of introducing a (pseudoscalar) Lagrange multiplier field $\Phi(x) \equiv (3/\kappa^2)^{1/2}b(x)$. Hence, the relevant torsion part of the quantum-gravity path integral $\mathcal{Z}$ includes a factor

$$ 
\begin{aligned}
\mathcal{Z} &= \int e^{-iS_G + iS_\psi} \propto \int D\mathbf{S} \, Db \exp \left[ i \int \left( \frac{3}{4\kappa^2} \mathbf{S} \wedge \mathbf{S}^* - \frac{3}{4} \mathbf{S} \wedge \mathbf{J}^5 + \left( \frac{3}{2\kappa^2} \right)^{1/2} b \mathbf{d}^*\mathbf{S} \right) \right] \\
&= \int Db \exp \left[ -i \int \left( \frac{1}{2} \mathbf{d}b \wedge \mathbf{d}b^* + \frac{1}{f_b} \mathbf{d}b \wedge \mathbf{J}^5 + \frac{1}{2f_b^2} \mathbf{J}^5 \wedge \mathbf{J}^5 \right) \right] \\
&= \int Db \exp \left[ -i \int \left( \frac{1}{2} \mathbf{d}b \wedge \mathbf{d}b^* - \frac{1}{f_b} bG(\mathbf{A}, \omega) + \frac{1}{2f_b^2} \mathbf{J}^5 \wedge \mathbf{J}^5 \right) \right],
\end{aligned}
\quad (7.57) $$

where $f_b = (3\kappa^2/8)^{-1/2} = \frac{M_P}{\sqrt{3\pi}}$, and in the last line we have partially integrated the second term in the exponent in the middle line of (7.57) and took into account the well-known field theoretic result that in QED the axial current is not conserved at the quantum level, due to anomalies, but its divergence is obtained by the one-loop result [853]: $\nabla_\mu J^5 = \frac{e^2}{8\pi^2} F^{\mu\nu} \tilde{F}_{\mu\nu} - \frac{1}{192\pi^2} R^{\mu\nu\rho\sigma} \tilde{R}_{\mu\nu\rho\sigma} \equiv G(\mathbf{A}, \omega)$, with the tilde denoting appropriate tensor duals.

An interesting example of a torsionful space-time which also provides a concrete UV-complete theoretical framework for quantum gravity comes from string theory. In string theories, torsion is introduced as a consequence of the existence of the (spin one) anti-symmetric tensor field $B_{\mu\nu} = -B_{\nu\mu}$ existing in the gravitational multiplet of the string. Indeed, as a result of the stringy “gauge” symmetry $B_{\mu\nu} \rightarrow B_{\mu\nu} + \partial_{[\mu}B_{\nu]}$, the low-energy string effective action depends only on the field strength $H_{\mu\nu\rho} = \partial_{[\mu}B_{\nu\rho]}$, where the symbol [...] denotes anti-symmetrization of the appropriate indices. In fact, it can be shown [854, 855] that the terms involving the field strength perturbatively to each order in the Regge slope parameter $\alpha'$ can be assembled in such a way that only torsionful Christoffel symbols, such as $\bar{\Gamma}_{\nu\rho}^{\mu} = \Gamma_{\nu\rho}^{\mu} + \frac{\kappa}{\sqrt{3}} H_{\nu\rho}^{\mu} \neq \bar{\Gamma}_{\rho\nu}^{\mu}$, appear in the low-energy effective action. In four dimensions, we may define the dual of $\mathbf{H}$, $Y_\sigma = -3\sqrt{2}\partial_\sigma b = \sqrt{-g} \epsilon_{\mu\nu\rho\sigma} H^{\mu\nu\rho}$, after adopting the normalizations of [852]. The field $b(x)$ is a form-valued pseudoscalar field, with a canonically normalized kinetic term which is termed the Kalb-Ramond (KR) axion [856], in order to distinguish it from other axion-like field coming, e.g. from the moduli sector of string theory. It plays a role entirely analogous to the gravitational axion field we have encountered above in our generic discussion of torsional field theories.

An important aspect of the coupling of the field $b(x)$ to the fermionic matter discussed above is its shift symmetry, characteristic of axion fields. Indeed, by shifting the field $b(x)$ by a constant, $b(x) \rightarrow b(x) + c$, the action (7.57) only changes by total derivative terms, such as $c R^{\mu\nu\rho\sigma} \tilde{R}_{\mu\nu\rho\sigma}$ or $c F^{\mu\nu} \tilde{F}_{\mu\nu}$. These terms are irrelevant for the equations of motion and the induced quantum dynamics, provided the fields fall off to zero sufficiently fast at infinity. The anomalous Majorana mass generation through torsion consists of augmenting the effective action (7.57) by terms that break the shift symmetry. To illustrate this last point, we first
<figure><img src="image_34.png" /><figcaption>**Figure 33.** Typical Feynman graph giving rise to anomalous fermion mass generation. The black circle denotes the operator $a(x) R_{\mu\nu\lambda\rho} \tilde{R}^{\mu\nu\lambda\rho}$ induced by torsion. (Similar to figure 1 in ref. [847].)</figcaption></figure>

couple the gravitational axion $b(x)$ to another pseudoscalar axion field $a(x)$. The proposed coupling occurs through kinetic mixing. To be specific, we consider the action

$$S = \int d^4 x \sqrt{-g} \left[ \frac{1}{2} (\partial_\mu b)^2 + \frac{b(x)}{192\pi^2 f_b} R^{\mu\nu\rho\sigma} \tilde{R}_{\mu\nu\rho\sigma} + \frac{1}{2f_b^2} J_\mu^5 J^{5\mu} \right. \\ \left. + \gamma(\partial_\mu b) (\partial^\mu a) + \frac{1}{2}(\partial_\mu a)^2 - y_a i a (\bar{\psi}_R^C \psi_R - \bar{\psi}_R \psi_R^C) \right], \quad (7.58)$$

where $\psi_R^C = (\psi_R)^C$ is the charge-conjugated right-handed fermion $\psi_R$, $J_\mu^5 = \bar{\psi}_R \gamma_\mu \gamma_5 \psi$ is the axial current of the four-component Majorana fermion $\psi = \psi_R + (\psi_R)^C$, and $\gamma$ is a real parameter to be constrained later. We have ignored gauge fields, which are not of interest to us, as well as the possibility of a non-perturbative mass $M_a$ for the pseudoscalar $a(x)$. Moreover, we remind the reader that the repulsive self-interaction fermion terms are due to the existence of torsion in the Einstein-Cartan theory. The Yukawa coupling $y_a$ of the axion moduli field $a$ to right-handed sterile neutrino $\psi_R$ may be due to non-perturbative effects. These break the shift symmetry, $a \to a+c$.

To evaluate the anomalous Majorana mass, it is convenient to diagonalize the axion kinetic terms by redefining the KR axion field, $b(x) \to b'(x) \equiv b(x) + \gamma a(x)$, and to appropriately redefine the axion field to appear with a canonically normalized kinetic term. In this way, the $b'$ field decouples and can be integrated out in the path integral, leaving behind an axion field $a(x)$ coupled both to matter fermions and to the operator $R^{\mu\nu\rho\sigma} \tilde{R}_{\mu\nu\rho\sigma}$, thereby playing the role of the torsion field. We observe through [847] that the approach is only valid for $|\gamma| < 1$, since otherwise the axion field would appear as a ghost, i.e. with the wrong sign of its kinetic terms, which would indicate an instability of the model. This is the only restriction of the parameter $\gamma$. The resulting effective action reads:

$$S_a = \int d^4 x \sqrt{-g} \left[ \frac{1}{2} (\partial_\mu a)^2 - \frac{\gamma a(x)}{192\pi^2 f_b \sqrt{1-\gamma^2}} R^{\mu\nu\rho\sigma} \tilde{R}_{\mu\nu\rho\sigma} \right. \\ \left. - \frac{i y_a}{\sqrt{1-\gamma^2}} a \left( \bar{\psi}_R^C \psi_R - \bar{\psi}_R \psi_R^C \right) + \frac{1}{2 f_b^2} J_\mu^5 J^{5\mu} \right]. \quad (7.59)$$

The mechanism for the anomalous Majorana mass generation is shown in figure 33. Adopting the effective field theory framework of [857], one may estimate the two-loop Majorana neutrino mass $M_M$ in torsional quantum gravity with an effective UV energy cut-off $\Lambda$ [847],

$$M_M \sim \frac{1}{(16\pi^2)^2} \frac{y_a \gamma \kappa^4 \Lambda^6}{192\pi^2 f_b (1-\gamma^2)} = \frac{\sqrt{3} y_a \gamma \kappa^5 \Lambda^6}{49152\sqrt{8} \pi^4 (1-\gamma^2)}. \quad (7.60)$$
In a UV complete theory such as strings, $\Lambda$ and $M_P$ are proportional to each other, since $\Lambda$ is proportional to the string scale $M_s = 1/\sqrt{\alpha'}$ ($\alpha'$ the Regge slope) and the latter is related to $M_P$ through $\frac{1}{g_s^2} M_s^2 V^{(c)} = \frac{M_P^2}{16\pi}$, with $g_s$ being the string coupling, and $V^{(c)}$ the compactification volume in units of $\alpha'$. It is interesting to provide a numerical estimate of the anomalously generated Majorana mass $M_M$. Assuming that $\gamma \ll 1$, the size of $M_M$ may be estimated from (7.60) to be $M_M \sim (3.1 \times 10^{11} \text{ GeV}) (\frac{y_a}{10^{-3}}) (\frac{\gamma}{10^{-1}}) (\frac{\Lambda}{2.4 \times 10^{18} \text{ GeV}})^6$.
Obviously, the generation of $M_M$ is highly model dependent. Taking, for example, the quantum gravity scale to be $\Lambda = 10^{17} \text{ GeV}$, we find that $M_M$ is at the TeV scale, for $y_a = 10^{-3}$ and $\gamma = 0.1$. However, if we take the quantum gravity scale to be close to the GUT scale, i.e. $\Lambda = 10^{16} \text{ GeV}$, we obtain a right-handed neutrino mass $M_M \sim 16 \text{ keV}$, for the choice $y_a = \gamma = 10^{-3}$. This is in the preferred ballpark region for the sterile neutrino $\psi_R \equiv \nu_R$ to qualify as a WDM [662], e.g. in the framework of the $\nu$MSM model [189], of interest in this white paper.

We conclude this subsection by noting that in a string-theoretic framework, many axions might exist that could mix with each other [858, 859]. Such a mixing can give rise to reduced UV sensitivity of the two-loop graph shown in figure 33. For instance, in models with three axion species, it is possible to obtain two-loop induced masses that are UV finite, and as such independent of the UV-cutoff $\Lambda$. For more details on neutrino mass estimates in such string-theoretic settings we refer the reader to [847].
# 8 Current and future keV neutrino search with astrophysical experiments

Section Editors:
Steen Hansen, Alexei Boyarsky

## 8.1 Previous bounds (authors: A. Boyarsky, J. Franse, A. Garzilli, D. Iakubovskyi)

When looking for fermionic DM particles, one should in general search above the Tremaine-Gunn limit [75] of about 3001eV). Essentially, below this mass, the phase space density of Dark Matter particles that would be required by the observed amount of Dark Matter in dwarf galaxies, would violate the Pauli-exclusion principle. If the mass is below $2m_ec^2$, such a fermion can decay to neutrinos and photons with energy $E_γ = \frac{1}{2}m_{DM}$ [322]. One can search for such particles in X-rays [566], either by aiming to detect the diffuse X-ray background [324] or by looking at nearby DM-dominated astrophysical objects [566]. As a side note, ref. [566] also proposed the stacking method which was later on involved in the detection of the 3.5 keV hint [190].

The omni-presence of Dark Matter in galaxies and galaxy clusters opens the way to check the decaying Dark Matter hypothesis [591]. The decaying Dark Matter signal is proportional to the column density $S_{DM} = \int \rho_{DM} d\ell$ — the integral along the line of sight of the DM density distribution (unlike the case of annihilating Dark Matter, where the signal is proportional to $\int \rho_{DM}^2 d\ell$). As long as the angular size of an object is larger than the field-of-view, the distance to the object drops out, which means that distant objects can give fluxes comparable to those of nearby ones [573, 574]. To be explicit, the flux in a Dark Matter decay signal is given by

$$F = \frac{M_{\text{fov}} \Gamma_{\text{DM}}}{4\pi D_L^2 m_{\text{DM}}} \qquad (8.1)$$

where $m_{DM}$ is the DM particle mass, $D_L$ the luminosity distance and $M_{fov}$ the amount of Dark Matter in the field of view of the telescope. The decay rate $\Gamma_{DM}$ is related to the particle lifetime as $1/\tau_{DM}$ and is given by

$$\tau_{DM} = \frac{1024\pi^4}{9\alpha G_F^2 \sin^2(2\theta)m_{DM}^5} = 7.2 \times 10^{29} \text{ sec} \left[ \frac{10^{-8}}{\sin^2(2\theta)} \right] \left[ \frac{1 \text{ keV}}{m_{DM}} \right]^5,$$

if DM is made of right-handed (sterile) neutrinos [177], with $\sin^2(2\theta)$ the interaction strength (mixing angle).

In the case of decaying DM, the signal strength also decreases more slowly with increasing distance from the centers of objects as compared to the case of annihilating DM where the expected signal is concentrated towards the centers of DM-dominated objects. This in principle allows one to check the Dark Matter origin of a signal by comparison between objects and/or by studying the angular dependence of the signal within one object, rather than trying to exclude all possible astrophysical explanations for each target [345, 567, 687].

The DM decay line is much narrower than the spectral resolution of the existing X-ray telescopes and, as previous searches have shown, should be rather weak. The X-ray spectra of astrophysical objects are crowded with weak atomic and instrumental lines, not all of which may be known. Therefore, even if the exposure of available observations continues to increase, it is hard to exclude an astrophysical or instrumental origin of any weak line found in the spectrum of individual object. However, if the same feature is present in the spectra of
many different objects, and its surface brightness and relative normalization between objects are consistent with the expected behavior of the DM signal, this can provide much more convincing evidence about its nature.

Many studies have indeed been performed on the widest range of objects. From objects as small as dwarf galaxies and as nearby as the center and halo of our own Milky Way, to other galaxies and clusters of galaxies, and more specialized searches such as through stacking of many objects or studying the peculiar 'dark blobs' like the Bullet cluster and A520 ([313, 324, 345-347, 485, 560, 566, 569, 581, 587-595, 598-602, 604, 606, 860]).

In reality, of course, after years of systematic search for this signal, any candidate line can be detected only at the edge of the possible sensitivity of the method. Therefore, one needs comparatively long-exposure data to cross-check the signal. Moreover, even an uncertainty at the level of a factor of a few in the expected signal, which is very hard to avoid, can result in the necessity to have significantly more statistics than in the initial data set in which the candidate signal was found. The parameter space that is available for a DM sterile neutrino within the νMSM is shown in figure 19.

## 8.2 X-ray telescopes and observation of the 3.55 keV line (authors: A. Boyarsky, J. Franse, A. Garzilli, D. Iakubovskyi)

Early in 2014, two independent groups reported a detection of an unidentified X-ray line at an energy of ~3.55 keV in the long-exposure X-ray observations of a number of Dark Matter-dominated objects: first in a stack of 73 galaxy clusters [190] and in the Andromeda galaxy and the Perseus galaxy cluster one week later [191].

The possibility that this spectral feature may be the signal from decaying Dark Matter has sparked a lot of interest in the community as the signal has passed a number of 'sanity checks' expected for Dark Matter decay: it is consistent with the expected mass scaling between the Andromeda galaxy, the Milky Way center and the upper bound from non-detection in the blank sky data [191, 608], and also between different sub-samples of clusters [190]. The signal has radial surface brightness profiles in the Perseus cluster, Andromeda galaxy [191] and in the Milky Way [608] that are consistent with our expectations for decaying Dark Matter and the mass distribution in these objects. Instrumental origins of this signal have been shown to be highly unlikely for a variety of reasons: the signal's presence in all of XMM-Newton's detectors and in Chandra's; the correct behavior with redshift (especially in the stacking procedure employed by ref. [190]); its absence in a very-long exposure blank-sky dataset. It has also not proved possible to attribute the signal plausibly and consistently between datasets to any known atomic emission line or lines [190, 191, 624, 625].

The center of our Galaxy is a classical target for DM searches. Its proximity allows the observations to concentrate on the very central part and therefore, one can expect a significant gain in the signal if the DM distribution in the Milky Way happens to be steeper than a cored profile. The Galactic Center (GC) region has been extensively studied with XMM-Newton and Chandra and several mega-seconds of raw exposure have been accumulated to date. Three recent studies [608-610] argued that this line may be a potassium K XVIII transition (as it is difficult to put an upper bound on the potassium abundance in the GC). However, as was argued in ref. [608], the correct question is: *Are the observations of the Galactic Center consistent with the Dark Matter interpretation of the 3.55 keV line of ref. [190, 191]?*

This is a non-trivial question, as the DM interpretation of the 3.55 keV line in M31 and the Perseus cluster puts a lower limit on the flux expected from the GC. On the other hand, the non-detection of any signal in the off-center observations of the Milky Way halo
(the blank sky dataset of ref. [191]) provides an upper limit on the possible flux from the GC, given observational constraints on the DM distribution in the Galaxy. Ref. [608] argued that the flux from the GC falls into this range, while a consistent interpretation of the line in all the datasets as an atomic transition is problematic [624, 625].

Even though the recent studies of stacked spectra of dwarf spheroidal satellites of the Milky Way by ref. [596], and galaxy clusters with Suzaku by ref. [612] and ref. [614] are in tension with the initial value of the DM lifetime from ref. [190] (which is based on their estimates of the DM content of their galaxy clusters), all of this work still leaves parameter-space for a common interpretation of all detections and non-detections in terms of DM. An overview of the detections and how they relate to each other and to the Dark Matter interpretation can be found in figure 34. Analyses by ref. [611] and ref. [613] make strong claims for ruling out the DM interpretation using sophisticated techniques. However, uncertainties in the DM content and distributions are still being debated [191, 608]; moreover, ref. [627] revealed problems in the morphological analysis of [613]. We have to refrain from making comment beyond this and would like to refer the reader to these references.

## Future searches for the 3.55 keV line

As mentioned in the previous section, the existing non-detections limit the range of DM-lifetimes associated with the 3.55 keV excess; specifically to $\tau_{DM} = (7.3 - 15.6) \times 10^{27}$ sec (green and magenta lines in figure 34). Given that the comparison of the results from different objects depends on assumptions about their DM content, the convincing ultimate test of the DM interpretation of the 3.55 keV signal requires an observation that allows for a detection of at least $3\sigma$ for the longest admissible lifetime $\tau_{DM} = 15.6 \times 10^{27}$ sec.

By the beginning of 2016, the forthcoming Astro-H [863] mission and the planned Micro-X sounding rocket experiment [864] were expected have sufficient spectral resolution to resolve the line against other nearby features and to detect the candidate line in the “strong line” regime [602]. In particular, the general hope was Astro-H should be able to resolve the Milky Way halo’s DM decay signal and therefore all its observations can be used, which unfortunately did not happen — see paragraph below.

The large field-of-view of the upcoming eROSITA X-ray observatory [865] will allow for tight constraints on the line by homogeneously covering nearby X-ray bright clusters to large radii. Cross-correlation power spectra of the cosmic X-ray background in the eROSITA all-sky survey will deliver additional competitive constraints [866].

## A short wrap-up of the sad Hitomi story

The Japanese Space Agency (JAXA) successfully launched the Astro-H satellite from Tanegashima Space Center in Japan on the 16th of February 2016. After it was placed in orbit and its solar panels deployed, the spacecraft was renamed Hitomi. However, JAXA lost contact with the satellite on March 26th, while it was executing its first test observations in orbit. Several sources, including the U.S. Joint Space Operation Center and various amateur observers, have reported that the satellite has broken up into several pieces of debris and changed orbit. In a press briefing on April 8th, JAXA stated that the Hitomi satellite appears to rotate with a period of 5.2 s, and that this rotation may have led to the separation of vulnerable pieces from the spacecraft's main body. It also confirmed that 10 other objects have been detected nearby on April 1st. Already at that time, it seemed possible that Hitomi (or Astro-H) would lost and could be recovered, which would make XMM-Newton and Chandra the only operational X-ray telescopes. In spite of the difficulties, the Hitomi
<figure><img src="image_35.png" /><figcaption>**Figure 34.** The flux in the 3.55 keV line in the spectra of the Galactic center, the Perseus cluster, and M31, as well as the $2\sigma$ upper bound from the blank sky dataset (from ref. [191]) as a function of the expected DM signal; see also ref. [313] which provides one of the strongest constraints on the line from the Chandra of M31. The results from ref. [190] are shown as the diagonal shaded strip (width $\pm 1\sigma$). The vertical sizes of the boxes are $\pm 1\sigma$ statistical error on the line's flux. The horizontal sizes of the boxes represent systematic errors in mass modeling: the projected mass density calculated using different literature models of the DM distribution. Diagonal lines correspond to different lifetimes of DM particles, assuming a decaying DM interpretation of the signal. Blue shaded regions give an example of the relation between the projected DM mass densities in the GC and in the blank-sky dataset based on the Milky Way mass model of ref. [861]. The magenta line shows the *estimated* sensitivity of 1.4 Msec of Draco observations [862]. The actually derived limits, however, differ [618, 619], see the discussion in section 5.3. The values of the lifetime above the *green line* ($\tau_{DM} = 7.3 \times 10^{27}$ sec) are excluded at $3\sigma$ level, based on ref. [596].</figcaption></figure>

team had been working hard to stabilize the satellite orbit with the hope of recovering part of the instrumentation. However, despite their best efforts, JAXA ceased efforts to rescue the satellite on April 28th, as it became clear that the spacecraft could not be recovered. From the updated information released by JAXA, it seems that the reason behind the failure was a too fast rotation that was caused unintentionally to compensate for a non-existing rotation in the opposite direction indicated by the software. This rotation was so fast that it took the satellite beyond its design limit, therefore causing main parts of the spacecraft to separate. See http://global.jaxa.jp/projects/sat/astro_h/ for more details. It is worth to note that, despite the satellite being lost, some data was collected from the Perseus cluster [631]; unfortunately, while yielding en even stronger constraint on a possible line signal, the statistics achieved had not been large enough to fully clear up the situation.
## 8.3 Lyman-α methods for keV-scale Dark Matter (authors: A. Boyarsky, J. Franse, A. Garzilli, D. Iakubovskyi)

The study of the matter power spectrum in the intergalactic medium (IGM) has been extensively used in the past to constrain the nature of DM — see for example [59, 326–332, 553], see also section 5.2. The IGM is constituted by the under-dense gas that fills the otherwise empty space between galaxies. Because the IGM is gravitationally dominated by the DM distribution, it is sensitive to the spatial distribution of DM; on the other hand its neutral fraction interacts with light. Hence, the IGM can be used as a probe of the DM power spectrum.

The physics of the IGM is relatively straight-forward with respect to other probes considered currently in cosmology, being constituted mostly by gas with almost primordial composition, and being organized in large scale structures that lie just outside the linear regime. Because of the relative simplicity of its physical description, the IGM is the natural choice for constraining cosmology, after the study of the cosmic microwave background. In particular, we are interested to get new limits on the nature of DM, from the observable of the IGM in the ultraviolet and optical bands, the so-called Lyman-α forest. This is the signature of the neutral component of the IGM on the spectra of distant quasars, and it is due to the scattering of the quasar’s light by the neutral hydrogen atoms through the Lyman-α transition. The ideal time for investigating the nature of DM through the Lyman-α forest is at high redshift, when the effect of AGN feedback is not as important as it can be at lower redshift.

Because we are interested in the smallest scales observable with the Lyman-α forest, where the difference between the CDM and the sterile neutrinos is the largest, we are susceptible to the unknown IGM temperature. In fact, the thermal history of the IGM is largely unknown, and in [538] the significant effect that arbitrary assumptions on the cosmic thermal history can have on SNe constraints have been made explicitly. Moreover, the authors have revised the current constraints on SNe, and we recognized that the latest analysis on high redshift and high resolution Lyman-α spectra, presented in [331], does not constrain the warmness of DM better than previous works (see [329] and [59]).

The temperature of the IGM has been measured from the broadening of the lines that constitute the Lyman-α forest, but one of the obstacles has been to discern between the size of the filaments that constitute the IGM, and the IGM temperature, as both the temperature and the physical extent of the clouds affect the broadening. In recent work [867], a new method for measuring both the size of the filaments and the temperature of the IGM has been outlined for the first time. Previous works on constraining the nature of DM through the Lyman α forest considered the flux power spectrum, in which the information about the initial matter power spectrum is mixed with astrophysical uncertainties. We believe that the line decomposition method considered in [867] can be adapted to the case of WDM cosmologies, and that in this way we can obtain new constraints on the nature of the DM, with an approach that is radically different from the one used in the previous works.

## 8.4 Pulsar kicks (author: S. Hansen)

When massive stars end their life in a supernova explosion a neutron star is born, and radio pulsars are a small fraction of the billions of neutron stars in the Milky Way. Observations show that these radio pulsars have large velocities, with averages of several hundred km/sec, and with a non-Gaussian tail extending up to 1600 km/sec [868, 869]. It is non-trivial to achieve such large velocities in standard core collapse supernova explosions [870]. In addition, any model predicting these large velocities should preferably also explain the observed correlation between the spin axis and the velocity of the pulsars [871, 872].
Standard neutrinos carry away up to 99% of the ~$10^{53}$erg of gravitational energy released in the collapse of the supernova, however, due to the large temperatures and densities, the neutrino scattering rate is in fact extremely high, and standard neutrinos diffuse spherically from the supernova core. Instead, if sterile neutrinos could be produced asymmetrically, then they would leave the supernova non-spherically. Indeed, with only a few percent asymmetry in the neutrino distribution, one could explain both the observed velocities and the correlation with the spin axis. The magnetic field inside the proto-neutron star provides exactly such an asymmetry by polarizing the electrons [873], and as we will describe below the combination of a large magnetic field and the mixing between a sterile neutrino with a $\mu$- or $\tau$-neutrino allows for a plausible explanation of both the pulsar velocities as well as the correlation with the spin axis [171].

The sterile neutrinos are created in the standard reactions

$$ \begin{aligned} p + e^{-} &\leftrightarrow n + \nu_{e}, \\ n + e^{+} &\leftrightarrow p + \bar{\nu}_{e}, \end{aligned} \qquad (8.2) $$

reduced by the mixing angle, $1/2 \times \sin^2(2\theta)$. In vacuum the production rate is negligible when the mixing angle is small, however, the matter-dependent potential may lead to the an MSW resonance, which implies that the active and sterile neutrinos are maximally mixed. This means that, for a given large magnetic field, there may exist a specific radius in the proto-neutron star where the temperature and density produce sterile neutrinos with a very specific momentum. And these neutrinos may then escape the supernova unhindered.

The surface of this resonance point is non-spherical due to the magnetic field, and it can be written as

$$ r_{\text{res}}(\phi) = r_0 + \delta \cos\phi, \qquad (8.3) $$

where $\phi$ is the angle between the neutrino momentum and the direction of the magnetic field and $\delta$ depends on the magnitude of the magnetic field and the chemical potential of the degenerate electron gas. The different momenta of sterile neutrinos are therefore created at different radii (with different temperatures) and this leads to an effective momentum transfer to the entire proto-neutron star and the asymmetry in the momentum distribution may be written [171]

$$ \frac{\Delta k_s}{k} = 0.01 \left( \frac{\mu_e}{100\text{MeV}} \right) \left( \frac{\mu_n}{80\text{MeV}} \right)^{1/2} \left( \frac{20\text{MeV}}{T} \right)^2 \left( \frac{B}{3 \times 10^{16}\text{G}} \right), \qquad (8.4) $$

where $\mu_e$ and $\mu_n$ are the chemical potentials of electrons and neutrons, respectively, while $T$ and $B$ are averaged core temperatures and magnetic field strengths.

Detailed calculations [874, 875] find that sterile neutrinos with masses about 5–20 keV and mixing angles in the range of $10^{-11} < \sin^2\theta < 5 \times 10^{-10}$ may provide the pulsar a sufficiently large kick, for proto-neutron star parameters in the range of $B > \text{few } 10^{16}\text{G}$, $T > 15\text{MeV}$, and central density $\rho > 10^{12}\text{g/cm}^3$.

An alternative non-resonant production mechanism was described in [876] (see also [874]) which allows one to kick the pulsars sufficiently for slightly smaller neutrino masses, between 2 and 10 keV, and larger mixing angles up to $\sin^2\theta \sim \times 10^{-7}$.

## 8.5 Supernovae (authors: S. Hansen and S. Zhou

When a massive star collapses to produce a supernova, the vast amount of gravitational energy is released mainly in neutrinos. During and immediately after the collapse the active
neutrinos are trapped in the core of the supernova and they diffuse out slowly on a timescale of the order 10 seconds. The observed duration of neutrino signals from SN1987A matched well with the theoretical predictions. However, if one of the active neutrinos is mixed with a sterile state, then the sterile neutrino may carry away too much of the energy, and thereby unduly shorten the neutrino burst. The discussion in this section uses exactly this energy-loss argument to constrain the allowed mass and mixing angle of a new sterile state.

### 8.5.1 The vacuum limit

When the supernova core emits energy in an invisible channel, the duration of neutrino signals shortens, potentially leading to disagreement with the observation of SN1987A. The result of [877] is that such exotic energy loss should obey

$$ \varepsilon < 10^{19} \text{ erg g}^{-1} \text{ s}^{-1}, \qquad (8.5) $$

where $\varepsilon$ must be calculated at typical values for the density and temperature of the supernova core, here taken as $\rho = 3 \times 10^{14} \text{ g cm}^{-3}$ and $T = 30 \text{ MeV}$.

If one of the active neutrinos is mixed with a sterile state, then the sterile neutrinos may be produced through both oscillations and frequent scattering with background particles, and carry away energies. The neutrino oscillation length in matter $\lambda_{\text{osc}}$ is much shorter than the mean free path $\lambda_{\text{mfp}}$ at typical supernova core temperatures [878]

$$ \frac{\lambda_{\text{mfp}}}{\lambda_{\text{osc}}} \approx 10^3 \left( \frac{\sin 2\theta}{10^{-4}} \right) \left( \frac{m_s}{10 \text{ keV}} \right)^2, \qquad (8.6) $$

which implies that once an active neutrino is produced, it will oscillate many times before collisions interrupt the coherent development of the flavor amplitude. For high mass sterile neutrinos, $m_s > 100 \text{ keV}$, the matter effects can be neglected. The sterile neutrino production rate is given by

$$ \Gamma_s = \frac{1}{2} \sin^2 2\theta \Gamma_a, \qquad (8.7) $$

where $\Gamma_a$ is the collision rate of active neutrinos on free nucleons. For $\nu_\mu$ or $\nu_\tau$, the neutral-current collisions are dominant and we obtain $\Gamma_a = G_F^2 E_\nu^2 n_B / \pi$ with $n_B$ being the number density of baryons. For $\nu_e$, the production and absorption on nucleons via the charged-current interaction are important, and the collision rate is $\Gamma_a = (C_V^2 + 3C_A^2) G_F^2 E_\nu^2 n_B / \pi$. In the case where one can ignore significant chemical potentials, the energy-loss rate for mixing with $\nu_\mu$ or $\nu_\tau$ becomes [352, 353, 878]

$$ \varepsilon_{\nu_s} = \left( \frac{\sin^2 2\theta}{3 \times 10^{-8}} \right) 10^{19} \text{ erg g}^{-1} \text{ s}^{-1}. \qquad (8.8) $$

Together with eq. (8.5) this leads to

$$ \sin^2 2\theta_{s\tau} < 3 \times 10^{-8}. \qquad (8.9) $$

For mixing with electron-neutrinos the calculation is slightly more complicated due to the chemical potential of the electron neutrinos, and the result is [879, 880]

$$ \sin^2 2\theta_{se} < 10^{-10}. \qquad (8.10) $$

The constraints above are valid for sterile neutrinos heavier than $m_s > 100 \text{ keV}$, where the matter effects can be neglected. However, they are only valid for masses smaller than
<figure><img src="image_36.png" /><figcaption>**Figure 35.** Constraints on the mixing parameters of a sterile neutrino mixed with a τ neutrino. The limits are rather similar for mixing with a μ neutrino. For mixing with electron neutrinos the vacuum limit (high mass) is stronger, $sin^22\theta < 10^{-10}$, however, the calculations are more involved and less accurately known for smaller masses ($m_s < 100$keV).</figcaption></figure>

$m_s < 100$ MeV: for typical core-collapse temperatures of 30 MeV, the average thermal neutrino energy is about 100 MeV, and thus heavier sterile states cannot be produced abundantly. For such massive neutrinos and maximal mixing the lifetime is about 10 μs, corresponding to distances of few km, i.e. of the order of the supernova core, and much longer than the mean free path of the active neutrinos of the order 10 meters. Thus, all the parameter space of massive neutrinos is excluded for mixing angles larger than the constraints above [353, 878]. This is shown as “vacuum limit” in figure 35.

### 8.5.2 Matter effects

The scattering of neutrinos off background particles will modify the dispersion relation of neutrinos in matter. This matter effect can be described by an effective potential, $V_{\nu\alpha}$ for each kind of active neutrinos. One therefore has an effective mixing angle which depends on the properties of the background

$$ \sin^2 2\theta_M = \frac{\sin^2 2\theta}{\sin^2 2\theta + (\cos 2\theta + E/E_r)^2} \quad (8.11) $$

where the resonant energy is defined as $E_r = \Delta m^2/2|V_{\nu\alpha}|$. The sign in the second term of the denominator is taken to be positive for neutrinos and a negative $V_{\nu\alpha}$, and it becomes negative either for antineutrinos or for a positive potential. For $\nu_\tau$, the resonant energy is given by [878]

$$ E_r = 3.25 \, \text{MeV} \left( \frac{m_s}{10 \, \text{keV}} \right)^2 \rho_{14}^{-1} |Y_0 - Y_{\nu_\tau}|^{-1}, \quad (8.12) $$
where $\rho_{14}$ is the matter density in units of $10^{14} \text{g cm}^{-3}$, and $Y_0 = (1 - Y_e - 2Y_{\nu_e})/4$ using $Y_x = (n_x - n_{\bar{x}})/n_B$, where $n_x$ and $n_{\bar{x}}$ stand for the number density of particles $x$ and that of their antiparticles, respectively. Given $Y_e = 0.3$ and $Y_{\nu_e} = 0.07$, the effective potential $V_{\nu_{\tau}}$ is negative, so the sign in the denominator in eq. (8.11) is positive and the resonant oscillations happen in the antineutrino sector.

For very large masses, $m_s > 100$ keV, the vacuum limit is obtained whereas for small masses, $m_s < 1$ keV the medium limit reached [879, 880], and the smooth transition between these two limits lies around 40 keV [324]. One effectively has a shift from the vacuum limit calculation of

$$ \sin^2 2\theta \rightarrow \sin^2 2\theta \times \left(\frac{m_s}{40\text{keV}}\right)^4, \qquad (8.13) $$

for small masses. The constraints in the matter dominated region is indicated by “medium limit” in figure 35.

With the help of eq. (8.11) one sees that the mixing angle in matter is always suppressed for neutrinos, whereas antineutrinos may experience matter enhancement when $E \sim E_r \cos 2\theta$. This happens for masses between 20 and 80 keV. The mixing angle of antineutrinos is always larger than that of the neutrinos, and hence the emission rate of the antineutrinos always exceeds that of the neutrinos. Hence, an asymmetry between the active neutrinos will therefore build up in the supernova core. Some of the effects hereof have been discussed [315, 878] and the conclusion is, that for masses near the resonance ($m_s \sim 50$ keV) the exclusion region increases by almost an order of magnitude for mixing with a tau neutrino. For mixing with electron neutrinos the calculation is significantly more complicated due to the high $\nu_e$ degeneracy and the large electron number. More explicitly, the resonant energy in this case is [315, 881]

$$ E_r = 13 \text{ MeV} \left(\frac{m_s}{10 \text{ keV}}\right)^2 \rho_{14}^{-1} |1 - 3Y_e - 4Y_{\nu_e}|^{-1}. \qquad (8.14) $$

The effective potential $V_{\nu_e}$ is initially positive for $Y_e = 0.3$ and $Y_{\nu_e} = 0.07$, indicating the resonance in the neutrino sector. However, as the lepton number decreases due to the emission of sterile neutrinos, the resonant energy is rapidly pushed to infinity corresponding to the condition $1 - 3Y_e - 4Y_{\nu_e} = 0$. As a consequence, the constraints in the vacuum limit become applicable to the cases of $m_s \ll 100$ keV as well [315, 881].

The mixing with electron neutrinos may also affect the explosion of the supernova itself. Intuitively one would expect that mixing would reduce the energy of the supernova, and hence make it even more difficult to make supernovae explode. Several papers have suggested alternatives, for instance through a reconversion which could transport energy from the deep core to the region just below the neutrino sphere [882]. This mechanism may be operational for masses of few keV. Alternatively, an anisotropic neutrino emission is also found to be able to enhance the energy of the exploding supernova [883].

An interesting possibility for boosting the supernova explosion includes a neutrino with a mass above the pion mass [354]. Such neutrinos will preferentially decay to a pion, which immediately decays to two photons [353] $\nu_s \rightarrow \nu_\alpha \pi^0 \rightarrow \nu_\alpha \gamma \gamma$. If the mixing angle is extremely small, such a decay may help re-ionize the Universe [884], however, if the mixing angle is just right, $\sin^2 2\theta \sim 10^{-7}$, then the sterile neutrinos may decay before leaving the supernova. In this way one may produce the sterile neutrino in the hot core of the supernova, and its decay may deposit energy near the shock front. Thereby one alters the supernova energetics which may increase the possibilities of a supernova explosion [354].
# 9 Current and future keV neutrino search with laboratory experiment

Section Editors:
Susanne Mertens, Loredana Gastaldo

## 9.1 Introduction

Direct dark matter experiments are an essential probe to complement indirect searches in a largely model-independent way. In analogy to WIMP searches, two distinct paths can be considered.

First, one may produce the dark matter particle in a laboratory setup and detect its presence via kinematic considerations. Section 9.2 focuses on this approach and discusses in detail a possible realization through tritium $\beta$ decay. Section 9.3 explores the sensitivity of this detection techniques with electron capture on holmium and other isotopes.

The second option is a direct detection of the dark matter particle present in our galaxy by using large-scale detectors. The density, energy distribution, and cross-sections of sterile neutrino dark matter is significantly different from WIMPs in typical cold dark matter scenarios. Nevertheless, the ability of existing direct DM search experiments (such as Xenon [36] and LUX [37]) to detect sterile neutrinos via elastic scattering is currently being investigated [885]. Alternatively, one may consider dedicated experiments directly detecting sterile neutrino dark matter for example via inverse $\beta$ decay. A detailed discussion of this approach is given in section 9.4.

Current laboratory limits for the mixing of sterile neutrinos with the electron neutrino flavour are of the order of $|U_{e4}|^2 \approx 10^{-3}$ in a large mass range of $m_4 \approx 1-100$ keV,$^{47}$ see figure 21. The experiments discussed here will provide much stronger sensitivity, possibly allowing to start probing the cosmological allowed region at $|U_{e4}| < 10^{-6}$. No current technology, however, allows to reach a sensitivity of the order of $|U_{e4}| \approx 10^{-10}$, which would allow to test indirect hints for sterile neutrinos of 7 keV mass, see section 5.3.

Finally, we consider the specific case of the $\nu$MSM [186, 188, 189], where a keV-scale sterile neutrino and two GeV-scale sterile neutrinos are proposed as a minimal extension of the Standard Model. We discuss the scientific reach of the proposed SHiP experiment at CERN to detect the latter heavy sterile neutrinos and thereby to probe the $\nu$MSM model.

## 9.2 Tritium beta decay experiments (author: S. Mertens)

Tritium $\beta$-decay is a platform to probe the existence of keV-scale massive sterile neutrinos with electron-flavor mixing. A small admixture of a keV-scale sterile neutrino mass eigenstate to the electron neutrino $\nu_e$ would manifest itself in high resolution $\beta$ spectroscopy: at a specific energy below the endpoint corresponding to the keV-scale sterile neutrino mass $m_s$ the reaction kinematics provides enough energy for emission of a heavy sterile neutrino mass eigenstate along with the electron [127, 886–888].

The $\beta$-decay spectrum is given as a superposition of the spectra corresponding to each mass eigenstate $m(\nu_i)$, weighted by its mixing amplitude $|U_{ei}|$ to the electron flavor. Since

<footnote>$^{47}$Throughout this section we assume the 4×4 or 2×2 mixing scenarios interchangeably. The first, considers the three active neutrino mass eigenstates $m_{1,2,3}$ plus one additional, mostly sterile, mass eigenstate $m_4$. The latter groups the light neutrino mass eigenstates together to a single effective neutrino mass $m(\nu_e)$ and assumes a single effective heavy neutrino mass $m_s$. In the case of 2 × 2 mixing the mixing amplitude can be written as $|U_{e4}|^2 = \sin^2 \theta$.</footnote>
the mass splittings between the three light mass eigenstates are so small, no current β-decay experiment can resolve them. Instead, a single effective light neutrino mass $m(\nu_e)^2 = \sum_{i=1}^3 |U_{ei}|^2 m(\nu_i)^2$ is assumed.

If the electron neutrino contains an admixture of a neutrino mass eigenstate with a mass $m_s$ in the keV range, the different mass eigenstates will no longer form one effective neutrino mass term. In this case, due to the large mass splitting, the superposition of the β-decay spectra corresponding to the light effective mass term $m(\nu_e)$ and the heavy mass eigenstate $m_s$, can be detectable. The differential spectrum can be written as

$$ \frac{d\Gamma}{dE} = \cos^2 \theta \frac{d\Gamma}{dE}(m(\nu_e)) + \sin^2 \theta \frac{d\Gamma}{dE}(m_s), \qquad (9.1) $$

where $\theta$ describes the active-sterile neutrino mixing, and predominantly determines the size of the effect on the spectral shape [127]. Figure 36 shows a qualitative example with perfect energy resolution and no energy smearing from atomic, thermal or scattering effects.

Tritium β decay provides distinct advantages when search for the signature of a keV-scale sterile neutrino. First, tritium β-decay is of super-allowed type, and therefore a precise theoretical description of the spectral shape is possible. Second, the 12.3-year half-life of tritium is relatively short, allowing for high signal rates with low source densities, which in turn minimizes source-related systematic effects such as inelastic scattering. Finally, with an endpoint energy of $E_0 = 18.575$ keV, the phase space of tritium provides access to a search for heavy sterile neutrinos in a mass range of astrophysical interest.

In this section the possibility of future experiments to measure the full phase space of tritium β-decay spectra will be discussed. The Troitsk neutrino mass experiment, which provided together with the Mainz Experiment the current best limits on the effective electron anti-neutrino mass [17, 128], is planning to utilize their apparatus to search for sterile neutrinos. The upcoming KATRIN Experiment, which is designed to probe the neutrinos mass with a sensitivity of 200 meV (90%CL) [129, 637] provides a ultra-luminous tritium source, allowing for a high statistics search for keV-scale sterile neutrinos. Promising novel detection techniques based on a measurement of the cyclotron radiation of the β electron or cryogenic techniques and the usage of an atomic tritium source (as opposed to molecular tritium) will further push the sensitivity of β-decay experiments and are successfully investigated by the Project 8 [889, 890] and Ptolemy [891] collaborations. Finally, we discuss the advantages and limitations of a full kinematic reconstruction [892] of β decay, which in principle would allow to detect a heavy sterile neutrino as missing energy in the decay.

### 9.2.1 The Troitsk experiment (authors: V.S. Pantuev, I.I. Tkachev, A.A. Nozik)

The “Troitsk nu-mass” laboratory is located in the Institute for Nuclear Research (INR) in Moscow, Troitsk, Russia. The initial intention of the experiment was to set up limits on the effective electron anti-neutrino mass or to measure it in tritium beta decay. This program has been conducted from 1985 to 2009 and resulted in the currently best upper limit [85, 128] on the effective electron anti-neutrino neutrino mass, $m(\nu_e) < 2.05$ eV. Later on the same data were used to search for sterile neutrinos with masses from 3 to 100 eV, see refs. [651, 893]. Limits on the mixing are shown in figure 38 as “Troitsk 2013”. Currently, the measurements of the β-spectrum are continued in INR in a much wider energy range with the intention to set up limits on sterile neutrinos in the keV mass range [894].

**Current experimental setup.** At the moment the “Troitsk nu-mass” installation is unique, as it is the only one in the world (before commissioning of KATRIN) combining
<figure><img src="image_37.png" /></figure>

<figure><img src="image_38.png" /><figcaption>**Figure 36.** (a): a tritium β-decay spectrum with no mixing (dashed black line) compared to a spectrum with a keV sterile neutrino mass of 10 keV and a mixing angle of sin²θ = 0.2 (solid red line). One can clearly see a kink-like signature of the keV-scale sterile neutrino at the electron energy E = E₀ − mₛ and its influence on the spectral shape below the kink energy. (b): ratio of a tritium β-decay spectrum without mixing and a spectrum with a 10 keV neutrino mass and a mixing amplitude of sin²θ = 10⁻⁷. The error bars correspond to a total statistics of ~10¹⁸ electrons.</figcaption></figure>

<figure><img src="image_39.png" /><figcaption>**Figure 37.** “Troitsk nu-mass” spectrometer.</figcaption></figure>

a windowless gaseous tritium source with an electrostatic spectrometer with adiabatic magnetic collimation. After a recent upgrade and the manufacturing of the new spectrometer, which has been commissioned in 2012, the best resolution of about 1-2 eV (depending on the settings) in the range up to 30 keV was obtained. The length of the spectrometer is about 10 meters, it is shown in figure 37.

The major systematic limitations at “Troitsk nu-mass” are related to:

* insufficiently accurate determination of the electron energy losses in the scattering of electrons inside the molecular tritium source;

* distortion of the spectrum, caused by the deviation of the transmission function of the spectrometer from the estimated or inaccurate account of the dependence on the energy efficiency of the registration;

* electronics dead time and pile up uncertainty;

* gas column density fluctuations;

* insufficient high voltage stability.
<figure><img src="image_40.png" /><figcaption>**Figure 38.** An estimate of sensitivity to the presence of additional sterile neutrinos depending on the neutrino mass at “Troitsk nu-mass”. The dot-dashed curve corresponds to measurements at the existing equipment, while the lower dotted curve is a projection for the result to be obtained after an anticipated upgrad. Gray lines correspond to the existing limits from figure 21.</figcaption></figure>

In order to achieve a statistical error comparable to the known systematics quoted for the existing equipment, Troitsk needs 15 days of live data acquisition with an integral source intensity of $10^6$ decays per second (which corresponds to the maximum count rate of $2 \cdot 10^4$ at 14 kV). The upper 95% C.L. on sterile neutrino mixing parameter, which can be achieved at the existing equipment with minimal modifications is shown in figure 38 by the dot-dashed red curve.

**Planned upgrades.** After these measurements are finalized, necessary upgrades to reach better sensitivity are planned at Troitsk. In particular this concerns:

* the absolute stability of the high voltage system will be better than 0.1 V;

* the relative stability of the intensity of the electron gun will be better than 0.1%;

* the relative accuracy of the calibration of the spectrometer transmission function will be better than 0.1%;

* accuracy of determination of the effective dead time of the data acquisition system will be of the order of 1 ns (at rates up to 200 kHz);

* the absolute accuracy of the determination of the effective thickness of the source will be of the order of or better than 0.001 in units of inelastic scattering probability;

* the intensity of the tritium source should be increased to get better statistics without loss of sensitivity at high rates.

The expected limits are approximately two orders of magnitude stronger than the existing ones, see figure 38, lower dotted curve.

Finally, in order to extend the probed range of neutrino masses, another new experiment is in preparation at Troitsk [895]. This new experiment will be employing the conventional
<figure><img src="image_41.png" /><figcaption>**Figure 39.** Main components of the KATRIN experimental setup. a: rear section, b: windowless gaseous tritium source, c: differential and cryogenic pumping section, d: prespectrometer, e: main spectrometer, f: focal plane detector.</figcaption></figure>

technique of proportional counters filled with tritium gas, but combine it with a modern fast signal digitization at high count rates — up to $10^6$ Hz.

### 9.2.2 The KATRIN experiment (authors: S. Mertens, J. Behrens, K. Dolde, V. Hannen, A. Huber, M. Korcekzek, T. Lasserre, D. Radford, P.C.-O. Ranitzsch, O. Rest, N. Steinbrink, C. Weinheimer)

The KATRIN experiment is a next-generation, large-scale, single $\beta$-decay experiment [637, 896]. It is currently under construction at the Tritium Laboratory at the Karlsruhe Institute of Technology (KIT) and will prospectively start taking data in 2016. The key goal of the experiment is to probe the effective light neutrino mass $m(\nu_e)$ with a sensitivity of 200 meV at 90% confidence level (CL) by analyzing the shape of the tritium $\beta$-spectrum in a narrow region below the $\beta$-decay endpoint energy, where the impact of the light neutrino mass is maximal.

Considering that only $10^{-13}$ of the $\beta$-decay electrons are created within an energy in the last 1 eV of the tritium $\beta$-decay spectrum, an extremely high decay rate is needed to reach the desired light neutrino mass sensitivity. KATRIN makes use of a gaseous molecular tritium source of very high activity ($\lambda_d \approx 1 \cdot 10^{11}$ decays per second) and stability (at the level of $< 10^{-3}$ per hour). These unique source properties may allow KATRIN to extend its physics reach to look for contributions of possible heavy neutrinos in the eV to multi-keV range. In the following we discuss the properties of the KATRIN main components (see figure 39) in the light of searching for keV-scale sterile neutrinos in the entire tritium $\beta$-decay spectrum.

**Advantages and limitations of KATRIN with respect to a keV-scale neutrino search.** The key component of KATRIN in view of the keV-scale sterile neutrino search is the high luminosity of its molecular tritium source. Its activity of $\lambda_d \approx 1 \cdot 10^{11}$ decays per second corresponds to a count rate of $\lambda_r \approx 1.5 \cdot 10^{10}$ counts per second (cps) (taking into account acceptance angle and the transmitted cross section of the source) and a total statistics of $N_{\text{decays}} \approx 1.4 \cdot 10^{18}$ electrons after 3 years of measurement time. Beyond that, the KATRIN source features high isotopic purity (95% T$_2$), which is constantly monitored by Laser Raman spectroscopy [897-899]. With temperature variations much smaller than 30 mK and a precise monitoring of the tritium column density [900, 901], the decay rate is expected to be extremely stable.

On the other hand, the gaseous tritium source entails a number of systematic effects. Energy losses of the $\beta$ electrons due to inelastic scattering in the source are unavoidable [902]. Furthermore, the source section poses magnetic traps for electrons emitted under a large angle with respect to the beam axis. These initially trapped electrons can escape eventually by
scattering into the beam. At the cost of reduced statistics, systematic effects due to scattering
may partly be reduced by lowering the source strength.

Another systematic effect related to the KATRIN setup is due to backscattering of β electrons and the emission of Auger electrons from the gold surface of the rear wall of the KATRIN setup [900]. Placing the rear wall in a low magnetic field would be a possibility to reduce the backscattering and the returning of backscattered electrons into the source. In depth study of these energy-dependent source and rear wall effects are ongoing.

KATRIN provides a sophisticated tritium retention system, reducing the tritium flow by 14 orders of magnitude from the windowless gaseous tritium source (WGTS) to the spectrometer. This system is based on differential [903] and cryogenic [904, 905] pumping obviating any tritium blocking membrane. This fact is extremely advantageous for the search for keV-scale sterile neutrinos, as it avoids systematic effects due to energy losses of the electrons in such a membrane. A scenario in which a detector would be installed at the rear section of KATRIN is disfavored since the tritium flow is not reduced as efficiently at this location.

The KATRIN main spectrometer works as a MAC-E filter, where a magnetic adiabatic collimation is combined with electrostatic filtering [129], providing an unsurpassed energy resolution of less than 1 eV at the endpoint energy and linearly decreasing with the electron kinetic energy. The basic working principle is to apply a certain retarding potential to the spectrometer, which allows only electrons with a kinetic energy higher than this potential to pass and reach a counting detector. Hence, by recording the count rate for different retarding potentials the *integral* shape of the energy spectrum is measured.

In the case of a keV-scale sterile neutrino search, the entire tritium β-decay spectrum
is of interest and therefore the main spectrometer would operate at very small retarding
energies to allow the electrons of the interesting part of the spectrum to reach the detec-
tor. To guarantee an adiabatic transport of electrons with high surplus energy through the
spectrometer [906], the magnetic field at the center of the spectrometer has to be increased
by a factor of ~3 while at the same time the magnetic field in the source section has to be
decreased by a factor of ~10.

One of the biggest challenges in using the present KATRIN experimental setup as
an apparatus to search for keV-scale sterile neutrinos arises from the high count rates of
$\lambda_r = 1.5 \cdot 10^{10}$ cps when aiming to measure the entire $\beta$-decay spectrum. The 148 pixel
silicon detector [907], as well as the electronics and DAQ systems, presently in use for normal
KATRIN operation, are not designed to deal with these electron rates.

**Pre-KATRIN measurement.** Despite the challenges listed in the previous section, a measurement with the first light of KATRIN might be feasible. The idea here is, to significantly reduce the tritium source strength, to achieve a counting rate of the order of 100 kHz which can be handled by the current KATRIN detector system. This reduction can be achieved in a 3-fold way: 1) less tritium is inserted into the WGTS, 2) the acceptance angle is reduced by reducing the source magnetic field relative to the maximal field, 3) the visible fluxtube is reduced by reducing the detector magnetic field.

Two measurement schemes are foreseen: 1) a differential measurement, making use of the current detector energy resolution of about 2 keV (FWHM) at 18.6 keV. In this case, the spectrometer would be set to a fixed low retarding potential at all times, allowing the entire part of the spectrum of interest to reach the detector. 2) An integral measurement, as in the standard KATRIN operation, could be performed, by measuring the count rate for different retarding potentials of the main spectrometer. The spectrometer would provide an sharp
(100 eV) energy cut-off. The latter requires high source stability whereas the first method requires a good understanding of the detector energy response. A combination of both will help reducing systematic uncertainties.

In general, for a low statistics Pre-KATRIN measurement the requirements on systematic uncertainties will less stringent, furthermore a major systematic effect related to inelastic scattering will be mitigated by reducing the source gas density. Figure 40 shows that from a statistical point of view, a 7-day measurement at reduced source strength will allow to significantly improve current laboratory limits. Detailed studies to assess the achievable sensitivity and to lay out a detailed measurement strategy are ongoing.

**Generic sensitivity studies for Post-KATRIN measurement.** Assuming the full KATRIN source strength being available for a keV-scale sterile neutrino search, a high statistical sensitivity can be reached. Figure 40 shows the statistical sensitivity assuming the number of tritium decays KATRIN would provide after three years of measurement time. However, before addressing the technical realization of such experiment generic sensitivity studies investigating the effect of theoretical and generic experimental uncertainties have been performed.

In [908] most importantly, the question whether theoretical uncertainties may impact on the sensitivity of the search was raised. By including the state-of-the-art theoretical corrections available in the literature and allowing for an normalization uncertainty in these corrections it was shown that the sensitivity is reduced by about a factor of five. This can be understood by the fact that the considered corrections are smooth functions of energy and therefore are not prone to mimic a sterile neutrino discontinuous signature. Furthermore, it could be shown that, by using the full information of the spectral shape, an energy resolution of 300 eV (FWHM) is sufficient to detect the signature of a sterile neutrino. This can be explained by the fact that the imprint of a sterile neutrino on the tritium β-decay spectrum is not only a local kink-like effect but it leads to a characteristic distortion of the spectrum in a rather large energy range below the “kink”.

In a second study [909] an alternate approach with the goal of detecting the characteristic kink-like signature of a sterile neutrino without perfect knowledge of the spectral shape was investigated. This approach is based on wavelet transformation, which detects which frequencies are present in a tritium β-decay spectrum at a given energy. Hence, in contrast to Fourier transformation, this method allows for both frequency and energy resolution. The studies revealed a high potential of this method, allowing to probe mixing angles of $sin^2θ > 10^{-6}$. However, in contrast to the spectral fit approach, a very good energy resolution of the order of 100 eV (FWHM) is necessary in order to not smear out the kink-like signal, which is what the wavelet technique is sensitive to.

**Novel detector system for post-KATRIN measurement.** To make use of the full KATRIN source strength, a novel detector system is needed. The combination of high rates and the smallness of the sterile neutrino signature make the design of a suitable detector system a very challenging effort. The main physics requirements are the following:

in order to minimize and control systematic effects (such as non-linearities, pile-up, energy-scale etc.) a sophisticated read-out system will be needed, possibly with a full digitization of each signal waveform. As a consequence, the number of pixels might be limited to < 100000. On the other hand, in order to minimize the rate per pixel and pile-up related systematic effects, the minimal number of pixels is > 10 000. In order to minimize charge sharing between pixels the minimal pixel size is 0.3 mm, which together with the number of pixels determines the minimal size of the detector.
<figure><img src="image_42.png" /><figcaption>**Figure 40.** 90% statistical exclusion limit of a Pre- and Post-KATRIN-like experiment. The Post-KATRIN measurement is based on a 3-years measurement with the full KATRIN source strength. The Pre-KATRIN measurement assumes a factor $10^5$ reduction of count rate and a measurement time of 7 days. The gray lines represent the current laboratory limits [642] and the parameter space excluded by astrophysical observations [397, 910].</figcaption></figure>

One of the most dangerous systematic effects will be related to backscattering of electrons from the detector surface. To mitigate this effect the detector location needs to be optimized. By placing the detector at a distance from the maximal magnetic field at the exit of the main spectrometer, most of the backscattered electrons are reflected back to the detector in short time intervals of less than a few hundreds of nanosecond. Since the magnetic flux area is increased at lower magnetic field, the detector area needs to be increased accordingly. A minimal detector radius is $r_{det} > 100$ mm. Furthermore, the energy threshold needs to be low enough (a few hundreds of eV) in order to detect backscattered electrons, which deposit only a small fraction of their energy in the detector. This in turn, requires a very thin deadlayer of the order of 10 nm.

To allow for a differential measurement of the tritium β-decay spectrum, a good energy resolution of 300 eV at 20 keV is necessary. This requires a thin deadlayer, low capacity and low leakage current. To combine the requirement of large pixel size with low capacity, a design with small (point-like) read-out contact and steering electrodes is being considered.

Currently, detector prototyping in collaboration with the Max-Planck Semiconductor Laboratory in Munich, Commissariat a l'énergie atomique et aux énergies alternatives (CEA), Lawrence Berkeley National Laboratory (LBNL), Oak Ridge National Laboratory, and Karl-sruhe Institute of Technology is ongoing.

**Time-of-Flight (TOF) mode.** An alternative to an upgrade of the detector system might be the introduction of a TOF mode. In a similar fashion to a novel detector, a TOF mode also changes the measurement strategy towards obtaining a differential spectrum. However, this approach is based on using the existing detector/DAQ infrastructure while applying minor modifications in the spectrometer section.

The principal sensitivity enhancement of such a strategy has already been shown in the context of light neutrino mass measurements in [911]. The basic idea is to measure and fit the TOF distribution of the electrons rather than the integrated energy spectrum.
<figure><img src="image_43.png" /></figure>

<figure><img src="image_44.png" /></figure>

**Figure 41.** (a): electron TOF as function of the surplus energy relative to the retarding potential for different starting polar angles and starting radius $r = 0$ m. The slope is especially higher for low surplus energies, meaning that a small change in energy will produce a significant change in TOF. Thus, the TOF spectrum gives information about the energy distribution close to the retarding potential. (b): TOF spectra for sterile neutrino with $m = 1$ keV, a fixed retarding potential of 17 keV and different mixing angles. The values of the mixing angles are strongly exaggerated to demonstrate the signature of a sterile neutrino in the TOF spectrum [908].

Since the TOF is dominantly a function of the electron surplus energy relative to the retarding potential as well as the polar emission angle of the electron, the measurement contains important information about the energy distribution closely to the retarding potential besides the raw count-rate (see figure 41). Having this supplementary information is statistically equivalent to a differential energy spectrum, thus enabling all the benefits of a differential measurement explained above as for instance better signal reconstruction of the 'kink' and increased tolerance against systematics.

As for the TOF measurement method, there are two methods being investigated right now. The first method is based on periodic blocking of the electron flux e.g. by switching the pre-spectrometer potential between two settings with full and zero transmission. If the pulses are sharp enough (in the order of a few micro seconds) and the time interval between the pulses is sufficiently large, the distribution of electron arrival times at the detector follows approximately the TOF spectrum. This method has already been tested in the past in the context of active neutrino mass measurements and proven technically realizable [912]. The shortcoming of it, however, is the tradeoff between statistics and TOF resolution. In order to have a good sensitivity on the actual TOF spectrum, the duty cycle needs to be low, resulting in loss of events. A too high duty cycle leads to a smearing of the TOF distribution. For small repetition intervals the danger of attributing electrons to the wrong cycle increases. On the other hand, if the existing detector/DAQ infrastructure is to be used, the count-rate needs to be lowered, which could be accomplished by using pulses with low duty cycle, giving sharp timing. Furthermore, positive effects concerning systematics are expected which could possibly compensate for the loss of statistics [913].

To address these issues, a second option is being investigated. In this strategy the time dependency is introduced by periodic post-acceleration and -deceleration. This process must be performed in a delay line behind the analysis plane at low magnetic field to avoid an effect
on the energy filtering. The arrival time of electrons at the detector then depends on the pulse phase and the electron velocity at the time of entry into the delay line. By this ‘time-focussing’ [914] it can be arranged, that for a specific electron energy, all electrons within one period are arriving approximately at the same time at the detector. To ensure the latter condition it is most likely beneficial to use the pre-spectrometer instead of the main spectrometer for the electrostatic energy analysis (as the energy resolution of the pre-spectrometer is sufficient for keV scale sterile neutrino search) and use the main spectrometer as delay line.

**Conclusion.** It seems feasible to operate KATRIN with changed source and electromagnetic design parameters in order to conduct a Pre-KATRIN keV-scale sterile neutrino search. This measurement has the potential to significantly improve current laboratory limits and will be provide valuable information for a Post-KATRIN measurement with higher statistics. Detector R&D, and simulations are currently being conducted to assess the feasibility and final sensitivity of the latter. A major goal is to ascertain whether a sensitivity of the order of $sin^2 \theta = 10^{-6}$ is possible with a KATRIN-like approach.

### 9.2.3 The project 8 experiment (author: B. Monreal)

A relativistic electron in a uniform magnetic field will emit cyclotron radiation. The frequency $f_γ$ of this radiation depends on the magnetic field strength $B$ and, due to a relativistic effect, on the electron kinetic energy $K$:

$$f_{\gamma} = \frac{f_c}{\gamma} = \frac{eB}{2\pi\gamma m_e},$$

where $γ = (1 + K/(m_ec^2))$ is the Lorentz factor and $f_c$ is the nonrelativistic electron cyclotron frequency, approximately 28 GHz per Tesla. This radiation is powerful enough that a modern cryogenic amplifier can detect single electrons. The relativistic shift allows us to use cyclotron-radiation measurements to measure $K$, which we refer to as cyclotron radiation emission spectroscopy (CRES). This was first proposed [889] for a very high resolution (sub-eV resolution) measurement of the tritium endpoint shape. The first CRES electron detections were obtained [890] by the Project 8 collaboration, which showed ~10 eV resolution on $^{83m}$Kr conversion electrons held in a magnetic bottle trap.

To search for keV-scale sterile neutrinos in the tritium spectrum, we expect to need the highest possible statistics but an energy resolution of only $ΔE_e \sim 100$ eV (~5 MHz/T when measured as frequency). The electron energy resolution of CRES is determined by several factors. One of these is the gas pressure in the source. Electron scattering on residual gas can be thought of as pressure-broadening of the cyclotron emission line, so we choose a density where the scattering rate is below the desired frequency resolution. The magnetic field needs to be uniform at a level of $ΔB/B = ΔE_e/m_e$; specifically, this precision applies to the time-averaged field experienced by trapped electrons (which are trapped along different B field lines and explore them with a range of different pitch angles).

Design parameters for such an experiment are shown in table 15. In contrast to an endpoint experiment, we will see numerous electrons radiating into each frequency bin at any time, unless the experiment is very highly segmented. This does not cause a “pileup”-like problem if we simply measure the average cyclotron radiation power as a function of frequency. Over a broad range of parameters, electron counting statistics (rather than, e.g., thermal noise statistics) are the main source of fluctuations, just like in a counting experiment. We can, of course, obtain a counting-experiment behavior by subdividing the tritium source into independently read-out subvolumes.
A mean-power measurement has notably different instrumental effects than a counting measurement. Frequency-dependent variations are expected in the noise temperature, cavity resonances, gain, and digitizer dynamic range. In a counting measurement, these effects may result in counting-efficiency corrections. In a power measurement, gain variations simply rescale the observed power. In either case, to measure the tritium spectrum in the presence of these variations, we need to exploit the variable B field. In a CRES experiment, true electron-related spectral features will move in frequency space when B changes. Gain shape variations and other equipment-dependent effects are fixed in frequency space and independent of B. Therefore, on top of whatever other gain calibrations are possible, a careful B scan ought to allow reconstruction of an instrument-independent measurement of the electron emission spectrum.

An electron may undergo inelastic scattering (losing 10–20 eV) but remain trapped and continue radiating at its lower energy, or it may undergo an angle change that allows it to escape from the magnetic mirror. The magnetic mirror configuration determines the range of pitch angles which are trapped, and hence the average number of energy-degraded electrons which contribute to the power spectrum. A strong mirror will retain an electron through a large number of scatters, resulting in a resolution function with a very long low-energy tail. In this case the instrument behaves somewhat like an integrating spectrometer. A weak mirror will allow for a narrower resolution function but at a cost in acceptance.

The scattering resolution function would itself need to be known very precisely in order to measure the overall tritium spectrum shape. In a sterile neutrino search, looking for a fairly sharp spectral discontinuity, the only requirement is that the resolution function varies smoothly with energy. We have not identified any likely sources of discontinuities in this function, but the question can be answered experimentally using higher-resolution, single-scatter-resolving measurements with CRES. More generally, any source of sharp energy-dependent behavior (in radiated power, scattering, trapping, non-tritium backgrounds, etc.) might be a source of systematic error; since the CRES technique is very new we do not have a complete understanding of such systematics.

### 9.2.4 PTOLEMY experiment (authors: B. Suerfu, C.G. Tully)

The Princeton Tritium Observatory for Light, Early-universe, Massive-neutrino Yield, PTOLEMY, proposes a novel search strategy that combines ultra-high resolution calorimetric-based methods with simultaneous high-precision side-band integration made possible through a multi-band spectroscopic MAC-E selection. The systematic uncertainties from knowledge of the energy resolution of the calorimetry limit the sensitivity in tritium β-decay spectrum analyses and make the need for high resolution information unavoidable for sterile neutrino mixings below $|U_{e1}|^2 \sim 10^{-6}$. Studies of the PTOLEMY multi-band spectroscopic selection with simulation achieve sterile neutrino search sensitivities down to $|U_{e1}|^2 \sim 10^{-8}$ in the mass range of 2–14 keV for a tritium exposure of 300 μg·years.

**Introduction.** PTOLEMY is a prototype cosmic neutrino background experiment located at the Princeton Plasma Physics Laboratory (PPPL) [891]. The energy of tritium β-decay electrons is measured with a high-resolution calorimeter with a target resolution of 0.15 eV at 100 eV. In a tritium end-point analysis, the 18.6 keV electrons from tritium are decelerated down to 100 eV following a 10$^{-2}$–10$^{-3}$ precision MAC-E filter. The electrons are guided into a thin absorber layer thermally coupled to a transition-edge sensor (TES) for calorimetric measurements. Assuming that an anomalous X-ray observation precedes the sterile neutrino
<table><caption>**Table 15.** Parameters of the search for a 7 keV sterile neutrino with a 100 eV resolution CRES instrument. “Source density” is the tritium density compatible with this resolution. “Bandwidth” is the required readout bandwidth for a single 100 bin. “Rate per bin” is the rate of detectable electrons in 100 eV bins at the energy of interest. “Occupancy” is the number of (new) electrons appearing to each sample, assuming a non-subdivided experiment. “Stat error, 1y” is the fractional statistical precision on the spectrum obtained in 1 y. We give explicit numbers for (a) small experiment which could be realized within a few years and (b) an ambitious experiment aimed at approaching astrophysically-motivated mixing angles.</caption><thead><tr><th></th><th>General case</th><th>Small experiment</th><th>Large experiment</th></tr></thead><tbody><tr><td>field <i>B</i></td><td></td><td>1 T</td><td>4 T</td></tr><tr><td>volume <i>v</i></td><td></td><td>0.01 m<sup>3</sup></td><td>10 m<sup>3</sup></td></tr><tr><td>pitch angle <i>θ</i><sub>m</sub></td><td></td><td>5°</td><td>5°</td></tr><tr><td>Source density, Bq/cm<sup>3</sup></td><td>2×10<sup>7</sup> T<sup>-1</sup></td><td>2×10<sup>7</sup></td><td>8×10<sup>7</sup></td></tr><tr><td>Bandwidth, MHz</td><td>5 · <sup><i>B</i></sup>&frasl;<sub><i>T</i></sub></td><td>5.0</td><td>20</td></tr><tr><td>rate, s<sup>-1</sup></td><td>8×10<sup>4</sup> <sup><i>Btv</i></sup>&frasl;<sub>cm<sup>3</sup>·s·T</sub>sin(<i>θ</i><sub>m</sub>)</td><td>7×10<sup>7</sup></td><td>3×10<sup>11</sup></td></tr><tr><td>Pileup occupancy</td><td>0.008 · <sup><i>v</i></sup>&frasl;<sub>cm<sup>3</sup></sub> · sin(<i>θ</i><sub>m</sub>)</td><td>7</td><td>7000</td></tr><tr><td>Stat error, 1 y</td><td>3.5×10<sup>-3</sup> / √ <sup><i>Btv</i>sin(<i>θ</i><sub>m</sub>)</sup>&frasl;<sub>cm<sup>3</sup>·T·y</sub></td><td>2.1×10<sup>-8</sup></td><td>3×10<sup>-10</sup></td></tr></tbody></table>

search in the tritium spectrum, the energy window of interest, centered on the kink in the tritium β-decay spectrum, is known in advance. The candidate sterile neutrino mass is twice the X-ray energy, for X-rays emitted in the rest frame of the sterile neutrino, and the energy resolutions achieved with X-ray observatories based on the TES-technology are typically a few eV [915].

This study investigates an approach to fit the tritium spectrum that can yield high sensitivity to low active-sterile mixing angles across a broad range of keV-scale sterile neutrino masses. The most sensitive signal shape information for the sterile neutrino search is within a narrow energy window in the vicinity of the predicted kink. The side-bands of the kink region provide predominantly normalization constraints which can be integrated at lower resolution. The separation of the electrons with energies in the vicinity of the kink region from the side-band region requires spectroscopic selection. Spectroscopic selection has a vital impact on the rates seen by calorimetry in the central energy window. In the central energy window, the $10^{-2}$ reduction in rate will allow calorimeter pixels to maintain a relatively slow readout rate, 10 kHz, compatible with ultra-high resolution TES. For a tritium mass of 100 µg, roughly $10^4$ high-resolution calorimeter cells are needed. The reduction of the number readout channels could be achieved with microwave multiplexing techniques, allocating MHz analog bandwidth per channel with resonance frequencies of a few GHz. The side-band calorimeter readout would benefit from windowless avalanche photo-diode (APD) detectors with 40 MHz readout, as currently used in the PTOLEMY prototype [891]. The resolution in the side-bands is expected to be roughly 300 eV. To meet rate requirements with low cell occupancy, approximately $10^3$ low-resolution calorimetry channels would be needed.

**Method.** Initial simulations of tritium endpoint electrons in a multi-band MAC-E filter show that spectroscopic selection is indeed possible. An example electron trajectory with toroidal coils and $E \times B$ drifting is plotted in figure 42. The magnetic adiabatic collimation
<figure><img src="image_45.png" /><figcaption>**Figure 42.** Spectroscopic multi-band MAC-E filter simulation of a tritium endpoint electron trajectory showing radial drift (on the left). On the right, a zoomed view of the same trajectory shows the cyclotron motion and magnetic bouncing near the coils.</figcaption></figure>

of the spiral trajectory of the β-decay electron drifts the electron from a high B field region in the inner torus to a low field outer torus. The phase space of the system grows radially, allowing for narrow selection of the energy band of interest within an azimuthal slice of the coil and electrode geometry. Details of the multi-band MAC-E spectroscopic selection are not described here. Instead, we assume that it will be possible to select a 100 eV energy window centered on a candidate sterile neutrino mass, based on mass information from an observation of an anomalous X-ray line. A mass uncertainty of 20 eV or smaller is sufficient for setting the energy window. We also assume that neighboring bands will be instrumented with lower resolution calorimetry with a stable and well-known efficiency to provide a high-precision side-band shape and integration. The purpose of this study is to examine whether the multi-band approach with high resolution calorimetry in the kink region has the sensitivity required to probe relevant mixing angles for WDM models.

The relative importance of calorimeter energy resolution against sources of energy smearing in the tritium decay and electron propagation to the calorimeter are considered. We take typical scales for atomic and Doppler shifting and assign a 5 eV energy smearing to the β-spectrum. We then compare the 5 eV smeared spectrum to an energy smearing of 50 eV, as shown side-by-side in figure 43. A smearing of 50 eV models the approximate performance of a low resolution calorimeter based on semi-conductor technology and limited by counting statistics from a large conduction-valence band-gap. This scale is also typical for the energy loss from inelastic electron scattering on tritium atoms or diatomic hydrogen. One can clearly see that fine features of the shape information are washed out in the rapidly falling region near the $Q - M_1$ endpoint. Large resolution smearing lowers the average signal-to-noise by spreading the sharp part of the signal over a broad range of energies.

The statistical uncertainties in the bins of the tritium β-decay spectrum are modeled directly, as opposed to throwing individual trials for on order $10^{18}$ events. This is achieved by constructing the underlying distribution, either with perfect resolution or convoluted with Gaussian energy resolutions, and then by finely binning the distribution into 0.1 eV bins in a 100 eV energy window centered around the kink. The height of the bin contents are then
<figure><img src="image_46.png" /><figcaption>**Figure 43.** A comparison of the tritium β-decay spectra with 5 eV (left) and 50 eV (right) energy smearing in the presence of a 10 keV sterile neutrino with a large mixing ($\sin^2 \theta = 0.05$) with electron neutrinos. The detailed shape information is washed out in the rapidly falling region near the $Q-M_1$ endpoint.</figcaption></figure>

smeared with a Gaussian counting statistics uncertainty corresponding to 300 µg.years of tritium exposure. We have not considered the exact atomic energy smearing and excitations in this study. In the PTOLEMY calorimetry-based approach, we plan to explicitly measure and unfold the atomic smearing distributions with sub-eV precision. In particular, we expect to be able to differentiate between the ~2 eV scale smearing present in diatomic tritium from the sub-eV smearing expected from weakly bound tritium on the surface of graphene through explicit measurement at the endpoint.

The direct measurement of the endpoint smearing with sub-eV resolution will reduce systematics on the knowledge of the signal shape for the heavy sterile neutrino search. Similarly, a narrow window shape analysis is less sensitive to β-decay spectrum shape variations that span many keV in electron energy, or correspondingly a few eV in outgoing $^3$He kinetic energy. The side-band integrals using lower resolution calorimetry will provide complementary sensitivity to the narrow window fit and with different systematics. Slowly varying shape information is the largest contributor to the sensitivity in the side-bands, especially when one considers that a single bin of 0.1 eV is roughly $10^{-5}$ of the full integration range. The step in the side-band normalization on either side of the narrow window is an important closure test to validate the self-consistency of the presence of a kink in the spectrum using independent data over the full phase space of heavy neutrino production from tritium β-decays.

**Result.** The narrow window fits for the heavy neutrino mixing as a function of sterile neutrino mass reproduce the input mixing parameters down to $\sin^2 \theta \sim 10^{-8}$ in conditions where the exact energy smearing is known. The fractional error on the fitted heavy neutrino mixing for these fits is plotted in figure 44 in conditions of perfect resolution. Over $3\sigma$ sensitivity is achieved at $\sin^2 \theta \sim 10^{-8}$ for sterile neutrino masses above 4 keV for 300 µg.years of tritium exposure. For finite resolution the results are degraded, as shown in figure 45, for energy resolutions of 5 eV and 50 eV. Over $3\sigma$ sensitivity is achieved at $\sin^2 \theta \sim 10^{-8}$ for sterile neutrino masses above 5 keV(8 keV) for an energy smearing of 5 eV(50 eV) in the case of 300 µg.years of tritium exposure.

One would argue that the degradation from 5 eV to 50 eV is not that significant relative to the order of magnitude sensitivity achieved in the mixing. However, the energy smearing
<figure><img src="image_47.png" /><figcaption>**Figure 44.** The fractional uncertainty on the fitted heavy neutrino mixing is plotted as a function of the mixing and heavy neutrino mass in conditions of perfect resolution. Over $3\sigma$ sensitivity is achieved at $\sin^2\theta \sim 10^{-8}$ for sterile neutrino masses above 4 keV for 300 µg-years of tritium exposure.</figcaption></figure>

<figure><img src="image_48.png" /><figcaption>**Figure 45.** A comparison of the fractional uncertainty on the fitted heavy neutrino mixing is plotted as a function of the mixing and heavy neutrino mass with 5 eV (a) and 50 eV (b) energy smearing.</figcaption></figure>

is known to be a leading systematic for such searches. If we vary the knowledge of the energy smearing by 10%, then the mixing sensitivity for 50 eV energy smearing deteriorates rapidly in the sterile neutrino search. Below $\sin^2\theta \sim 10^{-6}$, the extraction of the mixing parameter from the fit fails for 10% uncertainty on a 50 eV energy smearing, as shown in figure 46.

**Conclusion.** The general approach of using a narrow energy search window with high-resolution calorimetry measurements have been applied to the heavy sterile neutrino search in the vicinity of the expected kinematic edge. The results show promising sensitivity for resolutions of 5 eV in a 100 eV window when the energy smearing uncertainty is constrained at this precision by direct data measurements. Over $3\sigma$ sensitivity is achieved at $\sin^2\theta \sim 10^{-8}$ for sterile neutrino masses above 5 keV for 300 µg-years of tritium exposure.
<figure><img src="image_49.png" /><figcaption>**Figure 46.** The fractional difference on the fitted heavy neutrino mixing is plotted as a function of the mixing and heavy neutrino mass in conditions of 50 eV energy resolution and 10% uncertainty on knowledge of the energy smearing. Below $sin^2 θ \sim 10^{-6}$, the extraction of the mixing parameter from the fit fails.</figcaption></figure>

The information from side-band integration from lower resolution calorimetry and the slow variation in the shape predicted by heavy sterile neutrinos would provide important independent confirmation of the heavy sterile neutrino hypothesis over the full production phase space of the β-decay spectrum.

### 9.2.5 Full kinematic reconstruction of the beta decay (authors: F. Bezrukov, E. Otten)

The mass ($m_s$) of the unobserved sterile neutrino can be determined in principle from a single β-decay event, if one can reconstruct its energy and full momentum from the measured observables, namely the full momenta $\mathbf{k}$ and $\mathbf{p}$ of the emitted electron and the recoiling daughter, assuming that the parent nucleus was at rest [916–919].

$$m_s^2 = (Q - E_e - E_d)^2 - (\mathbf{p} + \mathbf{k})^2; \qquad (9.2)$$

where $E_e = \sqrt{m_e^2 + \mathbf{k}^2} - m_e$ and $E_d = \sqrt{M^2 + \mathbf{p}^2} - M$ are electron and recoil ion kinetic energies. The search should be limited to moderate neutrino energies not surpassing the expected mass by too large factors. On the other hand, the energy limit cuts back the phase space of the emitted neutrino and hence the rate of accepted events; it shrinks in proportion to the third power of the energy. This dilemma is particular painful in any attempt to search for the mass of the light neutrinos in β-decay, which so far have lead to an upper limit $m_ν < 2$ eV [17, 18]. This result — like all preceding ones — has been obtained from investigating the β-spectrum of tritium near its endpoint at 18.6 keV. The various ideas for determining the mass of the light neutrinos alternatively by measuring the full β-decay kinematics have not made it to the floor, yet. The difficulties of measuring energy and momentum of daughter and electron with uncertainty < eV, together with the rate problem near the endpoint, seem yet insurmountable.
In case of searching for sterile neutrino emission in tritium decay in the mass range of keV, however, the problems of reaching sufficiently small uncertainties of the observables, as well as that of avoiding high $\gamma$-factors of the neutrino, are relaxed by 3 orders of magnitude. Hence it makes sense to check the chances of a missing mass experiment under this aspect anew. The ColTRIMS technique (Cold Target Recoil Ion Momentum Spectroscopy), pioneered and reviewed by the atomic physics group at Frankfurt [920, 921] seems to offer a viable experimental ansatz, since it is capable of measuring the tiny recoil ion momentum occurring in tritium decay with sufficient precision. While for measuring the mass of an active, light neutrino precision of momenta measurements of order 1 eV or better is required, for the case of a keV neutrino precision of about 0.5 keV in momenta measurement would be sufficient. For the example of $^3$H decay this corresponds to a relative precision in momentum measurement of a few per mill.

There are a set of theoretical and experimental obstacles that limit the sensitivity of the full kinematic reconstruction technique. They concern the problem of luminosity and finite temperature of the source and the precision of the reconstruction of the event kinematics necessary to avoid contamination of the signal by active neutrino events. In the following we will present the principle components of a particular ColTRIMS scheme which might suit our purpose and analyze a few typical decay events with respect to the uncertainties of the results which are imposed by the instrument. We will then discuss the three major sources of systematic uncertainties of the final state kinematics which cause background from light neutrino events in the kinematic domain of heavy sterile ones. They stem from (i) the finite temperature of the source, (ii) the emission of an additional, unobserved $\gamma$-particle, and (iii) scattering of the recoiling daughter-ion within the source. Whereas the temperature problem seems manageable, the two others will cut back the sensitivity towards small mixing probabilities $\theta^2$ seriously.

**Cold $^3$H-source.** The luminosity problem arises in particular at small $\theta^2$ and requires either a rather large source or a rather dense source. The former choice will deteriorate the angular resolution of the momenta **k** and **p**, as well as the time of flight measurement (TOF) of the recoil ion by which the amount of its momentum is determined. The latter choice leads to enhanced scattering of the electron and in particular of the slow recoil ion from source atoms, affecting the angular distribution again.

Currently [920], one can build supersonic gas jets with particle densities of about $10^{11} - 10^{12} \text{ cm}^{-3}$ and internal temperature of the moving jet of the order of 0.1 K. Alternatively one may choose the well known optical traps either of the magneto-optical or of the electric dipole type reaching densities of $10^{10} \text{ cm}^{-3}$ and temperatures below 0.1 mK. These methods provide $10^6 - 10^8$ beta decays per year for the source size of about 1 mm$^3$. Still higher rates would be problematic in view of the rather wide TOF-window of the recoil ions (see figure 47), during which accidental coincidences of different decay events will occur. Part of them may fall into the kinematic domain of heavy neutrino decay and hence produce indistinguishable background events.

The ideal source material would be a gas of spin polarized atoms since they do not recombine. They could be trapped for instance by an electric dipole trap, i. e. within the focus of a laser cavity by the electric polarization potential $V = -\frac{1}{2}\alpha\mathbf{E}^2$ where $\alpha$ is here the electric polarizability and $\mathbf{E}$ the electric field strength of the laser field. Taking for $\alpha$ the estimate of the static polarizability of the hydrogen atom $\alpha_H = 4\pi\epsilon_0(9/2)a_0^2$ (in SI units, $a_0$ is the Bohr radius) one finds that a laser power of 100 kW, focused onto a focal spot of $10^{-4} \text{ cm}^2$ within the cavity of a CO$_2$ laser, would be sufficient for trapping a gas at a temperature of 0.01 K.
**Detectors for recoil-ions and electrons.** In view of the extremely small mixing angle as suggested by X-ray astronomy, one has to aim on optimized detection efficiency of the rare sterile neutrino events. The ColTRIMS technique allows projecting the total flux of the slow recoil ions onto a multichannel plate detector by help of a weak electric drift field. **p** is reconstructed from the coordinates of the hit channel and the time of flight with high precision thanks to the excellent angular and temporal resolution of this device. This option is not given, however, for the energetic and very fast decay electrons. Instead one has to look out for a detector array which offers sufficient angular and energy resolution as well as a reasonably large solid angle. Covering a fraction of of $\Delta\Omega/4\pi = 10\%$ of the full solid angle with angular resolution of $\simeq 0.5^\circ$ would require segmentation into $\simeq 6000$ pixels. In view of the marginal resolution of commercial items one may resort here to cutting edge R&D results. An appropriate detector might be built from the sophisticated calorimetric 64-pixel chips described in [922, 923]. At 6 keV particle energy they feature signals of 2 eV full half-width within 90 ns rise time. The electron signal would be fast enough as to determine the moment of decay with sufficient precision.

**Analysis of selected heavy and light neutrino events.** In order to get an impression about the precision of results expected from such devices we have analysed a few characteristic events (see figure 47). $E_e$ has been set to 2000(2) eV, close to the maximum of the $\beta$-spectrum, $m_s$ to 7 keV as suggested from X-ray astronomy, and the angle $\phi$ enclosed by **p** and **k** has been chosen to 0°, 90°, and 180° respectively. The time of flight of the daughter, TOF$_d$, has been calculated for a flight path of 0.2 m. Choosing a TOF-uncertainty $\sigma$(TOF$_d$) = 0.1 $\mu$s and an uncertainty $\sigma(\alpha) = 0.5^\circ$ for the angle $\phi$ enclosed by **p** and **k**, we calculate for the 7 keV sterile neutrino uncertainties $\sigma(m_s)$ as given in line 6 of figure 47. For the stretched events, given in row 3 and 5 of figure 47, $\sigma(m_s)$ is mainly due to $\sigma$(TOF$_d$). $\sigma(\mathbf{p}_e)$ is negligible in view of the high energy resolution of the envisaged electron detector. The mass uncertainties of 0.42 keV and 0.1 keV are satisfactory and could be improved further by choosing a longer flight path. However, stretched events are quite rare since the differential solid angle vanishes there in first order, whereas it maximizes at $\phi = 90^\circ$. But towards $\phi = 90^\circ$, $\sigma(m_s)$ rises up to 1.2 keV (row 4), since it is dominated there by $\sigma(\alpha)$.

In the lower half of figure 47 we have analyzed competing events with emission of light neutrinos, choosing $E_e = 2000(2)eV$ again and TOF$_d$-values as calculated in line 4. The before stretched events are now kinked by 8° and 11° degrees, respectively. But in the region of favorable rates around $\phi = 90^\circ$, the $\theta$-angle differs by only 1.9° corresponding to a distance of 3 pixels. Hence separation of these events can only work if ion optics is perfect and any scattering excluded.

**Background induced by source temperature and unobserved $\gamma$-emission.** We turn now to the discussion of the before mentioned three major effects which cause irreducible broadening of the momentum distribution such that the kinematic domains of the dominant light and the rare heavy neutrino decays might overlap. In this case one would face an indistinguishable background from light neutrino emission in the kinematic domain of heavy sterile neutrino emission. The first stems from the thermal velocity of the decaying nucleus; it has been treated already in an earlier publication (c. f. [892]). The second concerns the possible emission of a soft photon during the decay process, which escapes detection. If the decaying nucleus was moving with thermal velocity **v**, and a photon of momentum $\mathbf{k}_\gamma$ escaped detection, then the uncertainty of the determination of the neutrino mass induced by these
<figure><img src="image_50.png" /><figcaption>**Figure 48.** Estimate of the relative branching per keV for undetected photon emission imitating the neutrino mass $m_s$ in an event $^3H \to ^3He + e^+ + \bar{\nu}_e + \gamma$.</figcaption></figure>

been estimated and is plotted in figure 48. Within the interval $2 \text{ keV} < m_s < 16 \text{ keV}$ it drops from $10^{-4} \text{ keV}$ to $10^{-7} \text{ keV}$. For the most interesting case of a 7 keV sterile neutrino it amounts to about $10^{-5} \text{ keV}$. No attempt has been made up to date to study the possibility of reducing this background by efficient detection of this radiation tail. It would be very hard.

**Background by scattering of recoil ions from source atoms.** Another obstacle to the experiment is the large scattering probability of the slow recoil ions within the source. The total elastic cross section for $^3H$ on $^3H$ collisions is about $10^{-14} \text{ cm}^2$ [924]. A similar value will hold for $^3He^+$ on $^3H$. Assuming a source of e. g. $2 \cdot 10^{10} \text{ }^3H$-atoms and a 10% efficiency of the detector, one would collect about $10^8$ events per year which seems necessary for a competitive sterile neutrino search. Trapping this source by laser in a volume of order $10^{-6} \text{ cm}^3$ would present a column density of $2 \cdot 10^{14} \text{ cm}^2$, causing a fatal scattering probability of order 1. If one would observe $^3H$ decay from a jet source instead, the same number of atoms may occupy a 1000 times larger volume. This would ease the scattering problem by a factor 100 which is still insufficient for a sensitive search.

**Summary.** Todays experimental technology allows in principle to search for a missing mass of order keV in tritium beta decay by full kinematic reconstruction of the event, although such an undertaking would require a very extensive design and R&D phase in order to arrive at a functioning setup. Still the sensitivity for detecting a very small admixture of heavy neutrino events in the decay will be handicapped by residual shortcomings in the full kinematic reconstruction. They concern predominantly two effects: (i) the missing momentum of an unobserved photon accompanying the decay, (ii) the scattering of the recoil ions from source atoms.

## 9.3 Electron capture experiments (author: L. Gastaldo)

The study of the calorimetrically measured electron capture (EC) spectrum of $^{163}$Ho is presently a promising method for the investigation of the electron neutrino mass in the sub-eV range. With experiments based on $^{163}$Ho it appears possible to reach, for the electron
neutrino mass, the same sensitivity which tritium based experiments, as KATRIN [129, 896] can achieve for the electron anti-neutrino mass.

¹⁶³Ho is considered the best candidate among all nuclides undergoing electron capture processes to be used in an experiment for the investigation of the neutrino mass because of its extremely low energy available to the decay, $Q_{EC} = 2.833 \pm 0.030_{stat} \pm 0.015_{syst}$ keV [925, 926]. Such a low $Q_{EC}$ allows for a reasonable fraction of counts in the endpoint region of the spectrum to be analyzed for identifying effects due to a finite effective neutrino mass.

The idea to use the analysis of the calorimetric spectrum of ¹⁶³Ho to determine the electron neutrino mass was first proposed by De Rujula and Lusignoli in 1982 [927]. After about 30 years the feasibility of such an experiment was demonstrated by the ECHo collaboration [922, 928] by showing the possibility to perform high resolution measurements of the ¹⁶³Ho electron capture spectrum using low temperature metallic magnetic calorimeters [929]. Meanwhile there are three large international collaborations which aim to reach the sub-eV sensitivity on the electron neutrino mass by the analysis of high precision and high statistics EC spectrum of ¹⁶³Ho: the already mentioned “Electron Capture in ¹⁶³Ho” (ECHo) collaboration [922, 928], the “Electron Capture Decay of ¹⁶³Ho to Measure the Electron Neutrino Mass with sub-eV sensitivity” (HOLMES) collaboration [930] and the “Neutrino Mass via ¹⁶³Holmium Electron Capture Spectroscopy” (NuMECS) collaboration [931]. Intrinsic to the calorimetric measurement approach is the fact that all events occurring in the detectors give a measurable signal. Because of that, all these three experiments would have the possibility to analyze the data not only in the endpoint region to look for a deviation of the shape in respect to the case of massless neutrinos, but also over the full spectrum to look for a small kink which would be the signature of the existence of a fourth mass eigenstate, $m_4$, and therefore proving the existence of sterile neutrinos. In the following one of these experiments, ECHo, will be described in more details as well as the approach to investigate the existence of sterile neutrinos.

**¹⁶³Ho electron capture spectrum.** In an EC process a nucleus $Z^A X$ decays by capturing an electron from the inner atomic shells and emitting an electron neutrino to $Z_{-1}^A X$. Considering only the leading first order excitation, the daughter atom after the capture of the electron is left with one hole in the internal shell and one electron more in an external shell. Higher order excitations would include the presence of more than one holes in the atomic shells which happens with a much lower probability. The structures due to the higher order excitations could make the analysis of the spectrum more complicated both for the determination of the electron neutrino mass as well as for the identification of a kink due to a heavy neutrino mass state. Preliminary work to investigate the effects of higher order excitations in the analysis of the ¹⁶³Ho spectrum has already been done by the ECHo collaboration and will be discussed in the following paragraph. The atomic de-excitation is a complex process, which includes cascades of both x-rays and electron emissions (Auger electrons and Coster-Kronig transitions). The possibility to measure all the energy released in the decay minus the energy taken away by the neutrino simplifies the description of the spectrum. The expected shape of the calorimetrically measured EC spectrum, considering only first order excitations with only one hole in the internal shell, is:

$$ \frac{dN}{dE_C} = A(Q_{EC} - E_C)^2 \sqrt{1 - \frac{m_\nu^2}{(Q_{EC} - E_C)^2}} \sum C_H n_H B_H \phi_H^2(0) \frac{\frac{\Gamma_H}{2\pi}}{(E_C - E_H)^2 + \frac{\Gamma_H^2}{4}} \quad (9.5) $$

and shows Breit-Wigner resonances centered at an energy, $E_H$, where H indicates the level from which the electron has been captured, given in first approximation by the difference be-
<table><caption>**Table 16.** The binding energies $E_H^{\text{bin}}$ of the electron in Dy and the linewidths $\Gamma_H^{\text{lit}}$ as reported in [932].</caption><thead><tr><th>H</th><th>MI</th><th>MII</th><th>NI</th><th>NII</th><th>OI</th></tr></thead><tbody><tr><td><i>E</i><sub>H</sub><sup>bin</sup> [eV] [933]</td><td>2046.9</td><td>1844.6</td><td>420.3</td><td>340.6</td><td>49.9 [934]</td></tr><tr><td><i>Γ</i><sub>H</sub><sup>lit</sup> [eV] [935]</td><td>13.2</td><td>6.0</td><td>5.4</td><td>5.3</td><td>3.7 [936]</td></tr></tbody></table>

tween the energy of the electron that has been captured and the energy of the extra-electron in the outer shell, in respect to the daughter atom. The resonances have an intrinsic width $\Gamma_H$ related to the half-life of the excited states. The intensities of these lines are given mainly by the squared wave-function of the captured electron calculated at the nucleus $\phi_H^2(0)$, with a small correction, $B_H$, due to exchange and overlap. These factors are then multiplied by the nuclear shape factors $C_H$ and the fraction of occupancy of the H-atomic shell $n_H$. The Breit-Wigner resonances are then modulated by the phase space factor, which depends on the square of the electron neutrino mass $m(\nu_e)^2$ and the energy available to the decay $Q_{EC}$. A is a constant. In case of $^{163}\text{Ho}$, due to the extremely low energy available to the decay, only capture from the 3s and higher shells are allowed and due to the overlap of the electron wavefunctions with the nucleus, only electrons from the s and p$_{1/2}$ shells can be captured. Therefore the $^{163}\text{Ho}$ spectrum will consist of the following lines: MI (3s electrons), MII (3p$_{1/2}$ electrons), NI (4s electrons), NII (4p$_{1/2}$), OI (5s electrons), OII (5p$_{1/2}$ electrons) and PI (6s electrons). The binding energy of these electrons, $E_H^{\text{bin}}$, and the intrinsic linewidths $\Gamma_H^{\text{lit}}$ of the transition are given in table 16.

The electron neutrino mass that can be investigated with $^{163}\text{Ho}$-based experiments can be written in terms of the single mass eigenstates, in the scenario of only three active neutrinos, as:

$$m(\nu_e)^2 = \sum_{i=1}^{3} |U_{ei}^2| m(\nu_i)^2 \qquad (9.6)$$

In case keV sterile neutrinos would exist, at least a fourth neutrino mass eigenstate, $m_4$, would mix with the three “active” mass eigenstates to give the electron neutrino emitted in EC processes. In such a case, and considering the approximation that the three light neutrino mass eigenstates are much lighter than $m_4$, the electron neutrino state can be written as:

$$m(\nu_e)^2 = \sum_{i=1}^{3} |U_{ei}^2| m(\nu_i)^2 + |U_{e4}^2| m(\nu_4)^2 = |U_{ea}^2| m_a^2 + |U_{e4}^2| m_4^2 \qquad (9.7)$$

By substituting this equation for the electron neutrino mass in the formula of the spectrum and by using the approximation that $m_a = 0$ eV, the following expression is obtained:

$$\lambda_H = A \left( (Q_{EC} - E_H)^2 (1 - U_{e4}^2) + (Q_{EC} - E_H)^2 U_{e4}^2 \sqrt{1 - \frac{m_4^2}{(Q_{EC} - E_H)^2}} \right) C_H n_H B_H \phi_H^2(0) \qquad (9.8)$$

It follows that the evidence in the spectrum of the existence of the fourth neutrino mass eigenstate is a kink positioned at the energy $Q_{EC} - m_4$ and with an amplitude proportional to $|U_{e4}^2|$. With the analysis of the electron capture spectrum of $^{163}\text{Ho}$ it is possible to investigate the existence of sterile neutrinos only for $m_4$ smaller than the $Q_{EC}$ value of the decay of about 2.5 keV. Figure 49 shows the comparison between the expected calorimetrically measured $^{163}\text{Ho}$ spectrum calculated using Equation 9.8 in the case of no sterile neutrino (black dashed