<figure><img src="image_1.png" /></figure>

# Probing composite models at the LHC with exotic quarks production

Romain Kukla

► To cite this version:

Romain Kukla. Probing composite models at the LHC with exotic quarks production. Baldin ISHEPP XXIII - 23rd International Baldin Seminar on High Energy Physics Problems: Relativistic Nuclear Physics and Quantum Chromodynamics, Sep 2016, Dubna, Russia. pp.01009, 10.1051/epj-conf/201713801009 . hal-01555182

HAL Id: hal-01555182

https://hal.science/hal-01555182

Submitted on 1 Jun 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Probing composite models at the LHC with exotic quarks production

Romain Kukla<fnref n="1" />,<fnref n="2" />,*

**Abstract.** After the Higgs boson hunt, the LHC could be a powerful tool to unravel the mystery of which physics lies beyond the realm of the Standard Model. Different new sectors have been postulated to address naturalness: SUSY, extra dimensions and strong dynamics theories. Composite models extend EWSB to a global symmetry breaking whose pseudo-Goldstone boson is the SM Higgs boson. The resulting mass spectrum originates from a partial mixing between fundamental fermions and composite fields which creates massive states including new heavy quarks coupled preferentially to the top quark. Searches for these *top partners* have been carried out by the ATLAS and CMS collaborations, constraining the models. Other composite contributions are expected to enhance the 4-top production, which should be observable in the next years at the LHC.

## 1 Introduction

Even if the Standard Model (SM) seems to have reached its final form with the latest discoveries (1995: top quark, 2000: tau neutrino, 2012: Higgs boson), many open questions remain unsolved and physics Beyond the Standard Model (BSM) is necessary to answer them. For instance, the Higgs mass is stabilized in the SM thanks to very large quantum corrections from the loop diagrams. The hierarchy problem, or equivalently naturalness, is triggered by the existence of different scales in the SM especially between the fermion masses which should be at the same order of magnitude. The inclusion of dark matter candidates and strong CP violation to explain matter-antimatter asymmetry remain unsolved as for now. What is sure is that the SM needs to be extended and currently the two most explored ways to do so in the Higgs sector are: either considering that the Higgs boson is coupled with SUSY particles in the same way it is with the SM, with the help of superpartners which suppress the quadratic divergences of the Higgs mass and make the gauge couplings converging towards unification at high energy; or adding a strongly coupled theory which does not make the Higgs boson a fundamental object anymore and explains the fermion masses hierarchy with proto-Yukawa couplings depending on the linear mixing between the fundamental SM states and composite fields. In both cases, the top quark being the heaviest particle known in the SM and being coupled to the Higgs boson with a Yukawa coupling ~ 1, it is considered to be a logical probe to test BSM physics. Compositeness takes some inspiration from QCD pseudo-scalars which, as massive pseudo-Goldstone

<footnote n="1">¹Service de Physique des Particules, Institut de Recherche sur les lois Fondamentales de l'Univers, CEA Saclay, Gif sur Yvette, France</footnote>

<footnote n="2">²Centre de Physique des Particules de Marseille, UMR CNRS/Université d'Aix-Marseille, Marseille, France</footnote>

<footnote>*e-mail: romain.kukla@cern.ch</footnote>
bosons of the spontaneous chiral symmetry breaking $SU(2)_L \times SU(2)_R \to SU(2)_V$, have a lower mass (~130 MeV) compared to the typical QCD mass range around 1 GeV ($\rho$ mesons, neutron). This would be translated in terms of composite states around the TeV scale compared to a Higgs mass at ~125 GeV. Many realizations of this paradigm exist and are eagerly studied by the theoretical and experimental communities as they are a compelling alternative to supersymmetry. It has been pointed out in the very first developments of holographic Higgs theories that the AdS/CFT duality would map compositeness to strongly coupled physics, through Randall-Sundrum warped extra dimensions [1]. This paper will present the partial compositeness framework which is constrained in many experimental searches, of which we will discuss some of them recently published by the ATLAS collaboration. We will focus on the pair production of heavy quarks. A prospective study will conclude our overview of compositeness at the LHC with the proposal to probe intermediate production of heavy states in four top final states at $\sqrt{s} = 14$ TeV.

## 2 The compositeness paradigm

The extension of the SM is done by adding a composite sector, which is assumed to have a strongly-coupled dynamics, characterized by an exact global symmetry $G$ spontaneously broken into $\mathcal{H}$, where the electroweak group $G_{EW} = SU(2)_L \times U(1)_Y \in \mathcal{H}$. The $G$ group is supposed to be large enough so that there is at least 4 Goldstone bosons generated in the coset $G/\mathcal{H}$, which is usually done by choosing a $SO(5) \to SO(4)$ breaking. These four Goldstones bosons are defined as the Higgs doublet. In the elementary sector, which contains all other SM fields, $G$ is explicitly broken and the coupling between the two sectors transmits this to the composite sector where the Higgs doublet acquires a mass and a potential, thus triggering EWSB. This big picture is summarized in figure 1.

<figure><img src="image_3.png" /><figcaption>Figure 1. The composite Higgs paradigm realized with two sectors [2].</figcaption></figure>

More precisely, the couplings between the two sectors are chosen to be linear, so that the mass eigenstates are a mixing of elementary and composite fields. In addition to the common SM particles, heavy resonances are expected, as described in Equations 1-2, where $\phi_n$ denotes the mixing angle for each particle. This is the *partial compositeness* scenario in which the mass hierarchy is transposed in a mixing hierarchy: indeed, the Higgs boson being part of the composite sector, the Yukawa coupling to the SM particles is achieved through the proto-Yukawa coupling to composite states (which acquired mass by another mechanism) appearing in the mixing, see figure 2. Thus, the higher the mixing angle is, the heavier the particle is.

$$|\text{SM}_n\rangle = \cos \phi_n |\text{elementary}_n\rangle + \sin \phi_n |\text{composite}_n\rangle \quad (1)$$

$$|\text{Heavy}_n\rangle = -\sin \phi_n |\text{elementary}_n\rangle + \cos \phi_n |\text{composite}_n\rangle \quad (2)$$

More details on the theoretical side can be found in the literature, such as [2]. What is of interest for us is the phenomenology arising from this new sector which is composed, as explained, of the Higgs
<figure><img src="image_4.png" /><figcaption>**Figure 2.** The effective Yukawa coupling between SM fermions $f$ and the Higgs boson $H$ via their composite fraction $Q$ and $\tilde{T}$. The parameters $g_*$ and $\sin\theta$ characterize the proto-Yukawa coupling in the composite sector and the mixing between the two sectors [2].</figcaption></figure>

doublet plus heavy quarks coupled to the top (the light quarks are neglected due to tiny couplings and the composite bosons are not taken into account), written following the $5 = (2, 2)\oplus(1, 1)$ representation as a bi-doublet and a bi-singlet of the global symmetry: $Q = (2, 2)$ et $\tilde{T} = (1, 1)$, see Equation 3. The $T$, $B$ and $\tilde{t}_1$ have the same quantum numbers as $t_L$, $b_L$ and $t_R$. The heavy quarks doublet $\begin{pmatrix} T_{5/3} \\ T_{2/3} \end{pmatrix}$ (with respective charges of +5/3 and +2/3) and the singlet $\tilde{t}_2$ are consequences of the custodial symmetry introduced in $\mathcal{G}$ to stabilize the $\rho$ parameter (or $T$ operator) linked to $Zb_L\bar{b}_L$ vertex.

$$Q = \begin{bmatrix} T & T_{5/3} \\ B & T_{2/3} \end{bmatrix}, \quad \tilde{T} = [\tilde{t}_1, \tilde{t}_2], \quad H = \begin{bmatrix} \phi_0^* & \phi_0^+ \\ -\phi_- & \phi_0 \end{bmatrix} \quad (3)$$

These quarks couple to $t$ and $b$ via scalar and vector bosons:

* $T \to W^+b, T \to Zt$ and $T \to Ht$.

* $B \to W^-t, B \to Zb$ and $B \to Hb$.

* $T_{5/3} \to W^+t$.

* $T_{2/3} \to Zt$ and $T_{2/3} \to Ht$.

The main production diagram is the creation of a quark-antiquark pair initiated by QCD gluon fusion. An additional process leads to single production of these heavy quarks, highly dependent on the coupling, which can contribute at the same level of the pair production in the case of a very strong mixing parameter. These diagrams are shown in figure 3 for the $T_{5/3}$ case. For all the 4 heavy quarks, the pair production can give a peculiar signature with 2 same-charge leptons which is very rare in the SM, especially with associated $b$ quarks (we will see later that this is characteristic of 4-top events). We will now describe in more details experimental searches for heavy quarks coming from the composite sector and which couple to the top quark.

## 3 Searches for top partners at the LHC in the 2 same-sign leptons channel

The very first analysis based on these models was performed at the Tevatron by the CDF collaboration, using $2.7 \text{ fb}^{-1}$ of data collected at $\sqrt{s} = 0.98$ TeV [4]. The considered signal is the combined pair productions of $B$ and $T_{5/3}$, assumed to have the same mass, which give the same signature. After an event selection requiring at least 2 jets with $p_T \ge 15$ GeV, with one $b$-tagged, 2 same-sign leptons with $p_T \ge 20$ GeV and at least 20 GeV of transverse missing $p_T$ (nicknamed MET), they excluded
<figure><img src="image_5.png" /><figcaption>**Figure 3.** Inclusive final states for both pair and single productions of the $T_{5/3}$ exotic quark.</figcaption></figure>

mass regions under 365 GeV. This very first limit was superseded with the first LHC data [3], namely 4.7 fb⁻¹ at $\sqrt{s} = 7$ TeV, for similar cuts except a new one on the scalar sum of the $p_T$ of visible objects (jets, charged leptons): $H_T \ge 550$ GeV, justified by the intermediate production of two $T_{5/3}$ quarks whose mass is expected to be at least 350 GeV. This analysis used for the first time the single production for different regimes: very low and high couplings, so that its contribution ranged from marginal to dominant. The resulting exclusion plots from CDF and ATLAS are shown in figure 4.

<figure><img src="image_6.png" /><figcaption>**Figure 4.** Exclusion limit on the $T_{5/3}$ mass done by the CDF collaboration on B and $T_{5/3}$ pair productions [4] (left) and by the ATLAS collaboration on $T_{5/3}$ pair and single productions depending on the $\lambda$ coupling [3] (right).</figcaption></figure>

Two successive analyses were done by the ATLAS collaboration at $\sqrt{s} = 8$ and 13 TeV, respectively with 20.3 and 3.2 fb⁻¹ [5, 6]. For them, the main SM backgrounds leading to a real same-sign pair of leptons come from two different processes: the dibosons (WW, WZ, ZZ) and the vector boson production associated to a top quark pair ($t\bar{t}W, t\bar{t}Z$). Other processes can contribute marginally (WWW, WH, ZH, $t\bar{t}WW, t\bar{t}H$). The corresponding diagrams are shown in figure 5. They are described using Monte-Carlo samples including simulation of the ATLAS detector by Geant 4. The scarcity of the
final state makes instrumental effects much more important as a tiny probability of mis-reconstructing the charge of an electron, for instance, can lead to huge backgrounds due to large cross sections. This effect, called "mis-id", can turn a $\bar{t}\bar{t}$ event into a signal-like final state. Another instrumental source of background comes from semi-leptonic events with additional non-prompt leptons and fake leptons which originate either from secondary decays (jet, kaon, $\pi^{\pm}$) or from $\pi^{0}/\gamma$: we will call it shortly "fakes". The Monte-Carlo simulation cannot reproduce correctly these effects, they have to be estimated directly in the data using data-driven methods.

<figure><img src="image_7.png" /><figcaption>**Figure 5.** SM background processes giving 2 leptons of the same charge: dibosons (top) and $t\bar{t}V$ (bottom).</figcaption></figure>

Several methods are used by experimentalists, depending on the quality of the objects, the available statistics, the selection used. Concerning the latest result from ATLAS:

* **mis-id**: due to a large lever arm, $\mu$ mis-id is neglected. For electron, the charge flip rate is estimated on $Z \to e^{\pm}e^{\pm}$ in a control region under the Z peak using a likelihood extraction method after residual background subtraction. This rate is then applied on opposite-sign leptons events.

* **fakes**: After defining two selection qualities (loose/tight), the probability that a real/fake loose lepton passes the tight criteria is estimated with a likelihood matrix method in control regions enriched in fakes backgrounds. These real/fake rates are then used on same-sign data events.

<table><caption>**Table 1.** Signal regions categorization with respect to $H_T$, MET and b-jet multiplicity cuts.</caption><thead><tr><th colspan="4">e<sup>±</sup>e<sup>±</sup> + e<sup>±</sup>μ<sup>±</sup> + μ<sup>±</sup>μ<sup>±</sup> + eee + eeμ + eμμ + μμμ, N<sub>j</sub> ≥ 2</th></tr></thead><tbody><tr><td rowspan="3">400 &lt; H<sub>T</sub> &lt; 700 GeV</td><td>N<sub>b</sub> = 1</td><td rowspan="3">E<sub>T</sub><sup>miss</sup> &gt; 40 GeV</td><td>SRVLQ0</td></tr><tr><td>N<sub>b</sub> = 2</td><td>SRVLQ1</td></tr><tr><td>N<sub>b</sub> ≥ 3</td><td>SRVLQ2</td></tr><tr><td rowspan="6">H<sub>T</sub> ≥ 700 GeV</td><td>N<sub>b</sub> = 1</td><td>40 &lt; E<sub>T</sub><sup>miss</sup> &lt; 100 GeV</td><td>SRVLQ3</td></tr><tr><td>N<sub>b</sub> = 1</td><td>E<sub>T</sub><sup>miss</sup> ≥ 100 GeV</td><td>SRVLQ4</td></tr><tr><td>N<sub>b</sub> = 2</td><td>40 &lt; E<sub>T</sub><sup>miss</sup> &lt; 100 GeV</td><td>SRVLQ5</td></tr><tr><td>N<sub>b</sub> = 2</td><td>E<sub>T</sub><sup>miss</sup> ≥ 100 GeV</td><td>SRVLQ6</td></tr><tr><td>N<sub>b</sub> ≥ 3</td><td>E<sub>T</sub><sup>miss</sup> &gt; 40 GeV</td><td>SRVLQ7</td></tr></tbody></table>
The discriminant variable used to separate control and signal regions is $H_T$, as it is shown in figure 6. Background events with a $t\bar{t}$ pair peak around 300 GeV while signal samples, with a heavy quark pair, will peak around 1 TeV. On top of this variable, the b-jet multiplicity can be used to reject dibosons events; even $t\bar{t}$ events (from fakes and mis-id) can be rejected with a $n_b \ge 3$ cut. As a result, a set of orthogonal signal regions is chosen, summarized in table 1. Systematic errors were put on different sources of uncertainties: theoretical cross sections, selection efficiencies, luminosity. Fakes and mis-id data-driven methods have a 70% and 40% systematic error, respectively.

<figure><img src="image_8.png" /><figcaption>**Figure 6.** $H_T$ comparison between background processes and signals respectively at 8 and 13 TeV [5, 6].</figcaption></figure>

After comparison between data and background estimates in control regions showing good agreement, the signal regions were unblinded, giving the results plotted in figure 7. The 8 TeV analysis showed a 2 $\sigma$ excess in high b-jet multiplicity regions: in SRVLQ6, 4.2 $\pm$ 0.44 $\pm$ 1.16 events were expected, 12 data events were seen; for SRVLQ7, data gave 6 events against 1.14 $\pm$ 0.26 $\pm$ 0.34 for the expectation. The latest data at 13 TeV seem to argue for a statistical fluctuation as there is no excess anymore.

<figure><img src="image_9.png" /><figcaption>**Figure 7.** Background yields and observed data for each signal region in 8 TeV and 13 TeV analyses [5, 6].</figcaption></figure>
These results were interpreted as experimental constraints on the $T_{5/3}$ mass using the CLs method. The exclusion plots in figure 8 show the limits obtained for two data sets ($20.3 \text{ fb}^{-1}$ at 8 TeV and $3.2 \text{ fb}^{-1}$ at 13 TeV), the latest one being the most stringent one, excluding $m_{T_{5/3}} \le 990$ GeV. With such mass limits reaching the TeV scale, composite models predicting $T_{5/3}$ are starting to be challenged. Indeed, if we continue to push higher the mass of these top partners, a *little hierarchy* problem will appear: the exotic quarks proposed to solve the hierarchy discrepancies between u and t should be around the TeV, because of the Higgs mass. With $100 \text{ fb}^{-1}$ of data expected in the coming 2 years, one can expect to put very tight limits on these models and maybe jeopardize them.

<figure><img src="image_10.png" /><figcaption>Figure 8. Exclusion limits at 95 % C.L. on the $T_{5/3}$ mass using 8 TeV (left) and 13 TeV (right) data [5, 6].</figcaption></figure>

## 4 Four-top events at the LHC

The production of events with four tops in the final state occurs in the SM at a very low cross section ($15 \text{ fb} \times \sqrt{s} = 14 \text{ TeV}$). This process has not yet been observed at the LHC, the most precise searches having put upper limit on its cross section to 16 times the SM. In the context of compositeness, one can expect couplings between composite bosons and the top quark. This would add BSM contributions to the 4-top production and thus enhance the cross section [7]. We can model this by extending the SM as $SU(3)_c \times SU(2)_L \times U(1)_Y \times U(1)_X$, where $U(1)_X$ is linked to a symmetry breaking giving rise to a massive particle $Z'$ [8]. The mass of this generic resonance can be mapped to the scale of the new physics and its width is computed with Formula 4, with respect to the coupling $g_{tR}$ with the right-handed top [9, 10]. If this scale is too high, the process becomes a contact interaction between four top quarks. The corresponding diagrams are illustrated in figure 9. The final state $W^{\pm}W^{\pm}W^{\mp}W^{\mp}bbbb$ is very similar to the one studied for the pair production of $T_{5/3}$, which was $W^{\pm}W^{\pm}W^{\mp}W^{\mp}bb$.

$$ \Gamma = \frac{g_{tR}^2}{24\pi} (m_{Z'} - 2 \frac{m_I^2}{m_{Z'}^2}) \sqrt{1 - 4 \frac{m_I^2}{m_{Z'}^2}} \quad (4) $$

We try to estimate at 14 TeV the discovery potential in different leptonic channels for both SM and BSM signals, for $100 \text{ fb}^{-1}$, using Monte-Carlo simulations from MadGraph [11]: we scanned effective
scales of $\frac{\Lambda}{\sqrt{|C_{4t}|}} =0.5$, 1 and 1.5 TeV for $C_{4t} = 1$; and $Z'$ mass points of 1 and 2 TeV for a coupling $g_{IR} = 3$. The background taken into account in this analysis are: $t\bar{t}$, $t\bar{t}W$, $t\bar{t}Z$, $t\bar{t}H$. Dibosons samples were produced but as we require at least 3 b-jets, they are totally suppressed. BSM samples are simulated with interferences from the SM processes. The instrumental backgrounds are estimated on $t\bar{t}$ events using public results from ATLAS, for mis-id, and with the help of the DELPHES package for MadGraph which parametrizes the response of the detectors, for the fakes. The sensitivity is computed with $\frac{S}{VB}$ and the discovery (observation) luminosities correspond to $\sigma = 5$ (2).

<figure><img src="image_11.png" /><figcaption>**Figure 9.** SM (left), contact interaction (middle) and resonant BSM (right) production diagrams for 4-top events.</figcaption></figure>

The selection cuts, based on the variables $n_j$, $H_T$, MET and $n_b$, are optimized to maximize the significance. The figure 10 shows their distributions in the 2 same-sign leptons channel.

<figure><img src="image_12.png" /><figcaption>**Figure 10.** $H_T$, $n_{jet}$ and $n_{b-jet}$ distributions for the background (top) and for signals (bottom) at $\mathcal{L} = 100 \text{ fb}^{-1}$.</figcaption></figure>
Ongoing work shows that the most sensitive channel in terms of significance is the 2 same-sign leptons one, though the trilepton channel gives also very promising results. Even if the semi-leptonic channel is very bad for discovering 4-top events, it is very useful in order to study the top quark properties. Indeed, the leptonic W boson can be reconstructed from the lepton and the MET which comes mainly from the neutrino. By choosing the b-jet which minimizes the difference between the reconstructed top invariant mass and the pole mass, one can compute the 4-momenta of the leptonic top quark. The top quark polarization is then estimated using the angle $\theta_{lep}$ between the lepton in the top rest frame and the top in the laboratory frame [12]. As a result, the differential cross-section with respect to the polarization angle is given in Equation 5. In the SM, the top quark is produced in the same proportions in both chiralities, so A~0.5; in composite models, where the right-handed top has a privileged coupling, A is around 0.75.

$$ \frac{1}{\sigma} \frac{d\sigma}{d \cos \theta_{lep}} = \frac{1}{2} + (A - \frac{1}{2}) \cos \theta_{lep} \quad (5) $$

We reconstructed the leptonic top quark for SM processes and BSM 4-top production via a Z' resonance. The polarization angle distribution in figure 11 shows different slopes between standard tops and composite productions, as long as the Z' mass stays sufficient low so the SM contribution does not counterbalance the BSM slope. Fitting the slope on positive range improves the discrimination.

<figure><img src="image_13.png" /><figcaption>Figure 11. Top polarization for $t\bar{t}$, 4-top SM and BSM. A leptonic $p_T$ cut is done at 20 and 30 GeV.</figcaption></figure>

## 5 Conclusion

Composite models address the naturalness problem of the SM and solves the fine-tuning by considering the Higgs boson not as a fundamental object anymore. They predict the existence of heavy quarks which can be produced by pair at the LHC. Experimental searches have excluded mass ranges for the different top partners, especially in the 2 same-sign dilepton channel which is the most sensitive one, as the SM yields very few processes with this signature. Four-top events give a similar final state and can thus be searched in the same way. A prospective study showed that with optimized cuts, the SM process could be observed at $5\sigma$ using few hundreds of $fb^{-1}$ of data at $\sqrt{s} = 14$ TeV. BSM production involving composite models can already be probed with 13 TeV data. Top polarization measurement using the reconstruction of the leptonic top in the semileptonic channel showed a nice sensitivity to composite physics.
## Acknowledgments

The author is grateful to F. Deliot, J.-F. Laporte, A. Ouraou, L. Chevalier, C. Guyot, O. Mattelaer and J. Serra for fruitful discussions and crucial debugging help, as well to G. Servant for her guidance and financial support through the ERC Starting Grant Cosmo@LHC, without forgetting Professor E. Tomasi-Gustafsson, the Baldin seminar organization and JINR for their kind welcome in Dubna.

## References

[1] M. Son, R. Sundrum, R. Contino et T. Kramer, JHEP **0705**, 074 (2007)

[2] G. Panico and A. Wulzer, *The Composite Nambu-Goldstone Higgs* (Springer Lecture Notes in Physics, 2016) 1-316

[3] The ATLAS Collaboration, ATLAS-CONF-2012-130 (2012)

[4] The CDF Collaboration, Phys. Rev. Lett. **104**, 091801 (2010). FERMILAB-PUB-09-628-E

[5] The ATLAS Collaboration, JHEP **10**, 150 (2015)

[6] The ATLAS Collaboration, ATLAS-CONF-2016-032 (2016)

[7] G. Servant, Physics at the LHC 2010 Proceedings. DESY-PROC-2010-01/251

[8] G. Brooijmans et al., CERN-PH-TH-2010-096 (2009)

[9] L. Gauthier, CERN-THESIS-2012-157, PhD: U. Paris-Sud 11, Dept. Phys., Orsay: 2012-09-15 (2012)

[10] R. Kukla, CERN-THESIS-2016-136, PhD Orsay: 2016-06-17, NNT: 2016SACLS146 (2016)

[11] G. Servant et R. Kukla, to be published (2016)

[12] A. Pomarol and J. Serra, Phys. Rev. D **78**, 074026 (2008)