# Polarization-based Tests of Gravity with the Stochastic Gravitational-Wave Background

Thomas Callister, A.Sylvia Biscoveanu, Nelson Christensen, Maximiliano Isi, Andrew Matas, Olivier Minazzoli, Tania Regimbau, Mairi Sakellariadou, Jay Tasson, Eric Thrane

► To cite this version:

Thomas Callister, A.Sylvia Biscoveanu, Nelson Christensen, Maximiliano Isi, Andrew Matas, et al. Polarization-based Tests of Gravity with the Stochastic Gravitational-Wave Background. Physical Review X, 2017, 7 (4), pp.041058. 10.1103/PhysRevX.7.041058. hal-01669499

HAL Id: hal-01669499

https://hal.science/hal-01669499v1

Submitted on 24 Jul 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Polarization-Based Tests of Gravity with the Stochastic Gravitational-Wave Background

Thomas Callister,¹* A. Sylvia Biscoveanu,⁷,⁸ Nelson Christensen,²,³ Maximiliano Isi,¹ Andrew Matas,⁴ Olivier Minazzoli,³,⁵ Tania Regimbau,³ Mairi Sakellariadou,⁶ Jay Tasson,² and Eric Thrane⁷,⁸

¹LIGO Laboratory, California Institute of Technology, Pasadena, California 91125, USA

²Carleton College, Northfield, Minnesota 55057, USA

³Artemis, Université Côte d'Azur, Observatoire Côte d'Azur, CNRS, CS 34229, F-06304 Nice Cedex 4, France

⁴University of Minnesota, Minneapolis, Minnesota 55455, USA

⁵Centre Scientifique de Monaco, 8 Quai Antoine 1er, Monte Carlo 98000, Monaco

⁶Theoretical Particle Physics & Cosmology Group, Physics Dept., King's College London, University of London, Strand, London WC2R 2LS, United Kingdom

⁷School of Physics and Astronomy, Monash University, Clayton, Victoria 3800, Australia

⁸OzGrav: The ARC Centre of Excellence for Gravitational-Wave Discovery, Hawthorn, Victoria 3122, Australia

(Received 28 April 2017; revised manuscript received 18 August 2017; published 7 December 2017)

The direct observation of gravitational waves with Advanced LIGO and Advanced Virgo offers novel opportunities to test general relativity in strong-field, highly dynamical regimes. One such opportunity is the measurement of gravitational-wave polarizations. While general relativity predicts only two tensor gravitational-wave polarizations, general metric theories of gravity allow for up to four additional vector and scalar modes. The detection of these alternative polarizations would represent a clear violation of general relativity. The LIGO-Virgo detection of the binary black hole merger GW170814 has recently offered the first direct constraints on the polarization of gravitational waves. The current generation of ground-based detectors, however, is limited in its ability to sensitively determine the polarization content of transient gravitational-wave signals. Observation of the stochastic gravitational-wave background, in contrast, offers a means of directly measuring generic gravitational-wave polarizations. The stochastic background, arising from the superposition of many individually unresolvable gravitational-wave signals, may be detectable by Advanced LIGO at design sensitivity. In this paper, we present a Bayesian method with which to detect and characterize the polarization of the stochastic background. We explore prospects for estimating parameters of the background and quantify the limits that Advanced LIGO can place on vector and scalar polarizations in the absence of a detection. Finally, we investigate how the introduction of new terrestrial detectors like Advanced Virgo aid in our ability to detect or constrain alternative polarizations in the stochastic background. We find that, although the addition of Advanced Virgo does not notably improve detection prospects, it may dramatically improve our ability to estimate the parameters of backgrounds of mixed polarization.

DOI: 10.1103/PhysRevX.7.041058

Subject Areas: Astrophysics, Gravitation

# I. INTRODUCTION

The recent Advanced LIGO-Advanced Virgo observations of coalescing compact binaries have initiated the era of gravitational-wave astronomy [1–8]. Beyond their role as astrophysical messengers, gravitational waves offer unique opportunities to test gravity in previously unexplored regimes [9–12]. The direct detection of gravitational waves has already enabled novel experimental checks on

general relativity, placing the best model-independent dynamical bound to date on the graviton mass and limiting deviations of post-Newtonian coefficients from their predicted values, among other tests [5,6,8,13–15].

The measurement of gravitational-wave polarizations represents another avenue by which to test general relativity. While general relativity allows for the existence of only two gravitational-wave polarizations (the tensor plus and cross modes), general metric theories of gravity may allow for up to four additional polarizations: the x and y vector modes and the breathing and longitudinal scalar modes [9,10,12]. The effects of all six polarizations on a ring of freely falling test particles are shown in Fig. 1. The detection of these alternative polarization modes would represent a clear violation of general relativity, while their nondetection may serve to experimentally constrain extended theories of gravity.

<footnote>*tcallist@caltech.edu</footnote>

<footnote>Published by the American Physical Society under the terms of the Creative Commons Attribution 4.0 International license. Further distribution of this work must maintain attribution to the author(s) and the published article's title, journal citation, and DOI.</footnote>
<figure><img src="image_2.png" /><figcaption>FIG. 1. Deformation of a ring of freely falling test particles under the six gravitational-wave polarizations allowed in general metric theories of gravity. Each wave is assumed to propagate in the *z* direction (out of the page for the plus, cross, and breathing modes; to the right for the vector-*x*, vector-*y*, and longitudinal modes). While general relativity allows only for two tensor polarizations (plus and cross), alternate theories allow for two vector (*x* and *y*) and/or two scalar (breathing and longitudinal) polarization modes.</figcaption></figure>

Few experimental constraints exist on the polarization of gravitational waves [13]. Very recently, though, the simultaneous detection of GW170814 with the Advanced LIGO and Virgo detectors has allowed for the first direct study of a gravitational wave's polarization [7,16]. When analyzed with models assuming pure tensor, pure vector, and pure scalar polarization, GW170814 significantly favored the purely tensor model over either alternative [7,16]. This result represents a significant first step in polarization-based tests of gravity. Further tests with additional detectors, though, will be needed to sensitively test general relativity and its alternatives.

In particular, many alternate theories of gravity yield signals of mixed polarizations, yielding vector and/or scalar modes in addition to standard tensor polarizations. When allowing generically for all six polarization modes, the three-detector Advanced LIGO-Virgo network is generally unable to distinguish the polarization of transient gravitational-wave signals, like those from binary black holes [5,11–13,16,17]. First, the two LIGO detectors are nearly co-oriented, leaving Advanced LIGO largely sensitive only to a single polarization mode [5,7,12,13]. Second, even if the LIGO detectors were more favorably oriented, a network of at least six detectors is generically required to uniquely determine the polarization content of a gravitational-wave transient [11,12,18]. Some progress can be made via the construction of “null streams” [18], but this method is infeasible at present without an independent measure of a gravitational wave’s source position (such as an electromagnetic counterpart). Future detectors like KAGRA [19] or LIGO-India [20] will therefore be necessary to break existing degeneracies and confidently distinguish vector or scalar polarizations in gravitational-wave transients. It should be noted that the scalar longitudinal and breathing modes induce perfectly degenerate responses in quadrupolar detectors like Advanced LIGO and Virgo [12]. Thus, network of quadrupolar can at most measure five independent polarization degrees of freedom [12,16,18].

Beyond the direct detection of binary coalescences, another target for current and future detectors is the observation of the astrophysical stochastic gravitational-wave background, formed via the superposition of all

gravitational-wave sources that are too weak or too distant to individually resolve [17,21–25]. Although the strength of the background remains highly uncertain, it is expected to be detected by Advanced LIGO and Advanced Virgo at their design sensitivities [25–28]. Unlike direct searches for compact binaries, Advanced LIGO searches for long-lived sources like the stochastic background and rotating neutron stars are currently capable of directly measuring generic gravitational-wave polarizations without the introduction of additional detectors or identification of an electromagnetic counterpart [17,29–34]. The observation of the stochastic background would therefore enable novel checks on general relativity not possible with transient searches using the current generation of gravitational-wave detectors.

In this paper, we explore the means by which Advanced LIGO can detect and identify alternative polarizations in the stochastic background. First, in Sec. II, we consider possible theorized sources which might produce a background of alternative polarizations. We note, though, that stochastic searches are largely unmodeled, requiring few assumptions about potential sources or theories giving rise to alternative polarization modes (see, however, Sec. VI).

In Sec. III, we discuss the tools used for detecting the stochastic background and compare the efficacy of standard methods with those optimized for alternative polarizations. In Sec. IV, we then propose a Bayesian method with which to both detect generically polarized backgrounds and determine if alternative polarization modes are present. Next, in Sec. V, we explore prospects for estimating the polarization content of the stochastic background. We quantify the limits that Advanced LIGO can place on the presence of alternative polarizations in the stochastic background, limits which may be translated into constraints on specific alternative theories of gravity.

As new detectors are brought on-line in the coming years, searches for alternative polarizations in the stochastic background will become ever more sensitive. In both Secs. IV and V, we therefore investigate how the addition of Advanced Virgo improves our ability to detect or constrain backgrounds of alternative polarizations. Finally, in Sec. VI, we ask if our proposed search is robust
against unexpectedly complex backgrounds of standard tensor polarizations.

## II. EXTENDED THEORIES OF GRAVITY AND ALTERNATIVE POLARIZATION MODES

Searches for the stochastic background are largely unmodeled, making minimal assumptions about the source of a measured background. Nevertheless, it is interesting to consider which sources might give rise to a detectable background of alternative polarization modes. In this section, we briefly consider several possibilities that have been proposed in the literature. We focus mainly on scalar-tensor theories, which predict both tensor- and scalar-polarized gravitational waves [35]. Our discussion below is not meant to be exhaustive; there may well exist additional sources that can give rise to backgrounds of extra polarization modes. In particular, we do not discuss possible sources of vector modes, predicted by various alternative theories of gravity (see Ref. [29] and references therein). Note that, while advanced detectors may not be sensitive to the sources described below, these sources may become increasingly relevant for third generation detectors (or beyond).

Core-collapse supernovae (CCSNe) represent one potential source of scalar gravitational waves. Although spherically symmetric stellar collapses do not radiate gravitational waves in general relativity, they do emit scalar breathing modes in canonical scalar-tensor theories. While the direct observation of gravitational waves from CCSNe is expected to place strong constraints on scalar-tensor theories [36], only supernovae within the Milky Way are likely to be directly detectable using current instruments [37,38]. Such events are rare, occurring at a rate of $(0.6–10.5) \times 10^{-2} \text{ yr}^{-1}$ [39]. The stochastic gravitational-wave background, on the other hand, is dominated by distant undetected sources, and so in principle it is possible that a CCSNe background of breathing modes could be detected before the observation of a single Galactic supernova [40,41]. However, realistic simulations of monopole emission from CCSNe predict only weak scalar emission [36]. Nevertheless, certain extreme phenomenological supernova models predict gravitational radiation many orders of magnitude stronger than in more conventional models [37]. According to such models, CCSNe may contribute non-negligibly to the stochastic background.

Compact binary coalescences may also contribute to a stochastic background of scalar gravitational waves. In many scalar-tensor theories, bodies may carry a “scalar charge” that sources the emission of scalar gravitational waves [42,43]. Monopole scalar radiation is suppressed due to conservation of scalar charge, but in a general scalar-tensor theory there is generally no conservation law suppressing dipole radiation. Scalar-dipole radiation from compact binaries is enhanced by a factor of $(v/c)^{-2}$ relative to ordinary quadrupole tensor radiation (where v is the

orbital velocity of the binary and c the speed of light), and thus represents a potentially promising source of scalar gravitational waves. Electromagnetic observations of binary neutron stars place stringent constraints on anomalous energy loss beyond that predicted by general relativity; these constraints may be translated into a strong limit on the presence of additional scalar-dipole radiation [44,45]. Such limits, though, are strongly model dependent, assuming a *priori* only small deviations from general relativity. Additionally, pure vacuum solutions like binary black holes are not necessarily subject to these constraints. If, for example, the scalar field interacts with curvature only through a linear coupling to the Gauss-Bonnet term, scalar radiation is produced by binary black holes but not by binary neutron stars [46,47]. Alternatively, binary black holes can avoid the no-hair theorem and obtain a scalar charge if moving through a time-dependent or spatially varying background scalar field [48,49].

A variety of exotic sources may generically contribute to stochastic backgrounds of alternative polarizations as well. Cosmic strings, for instance, generically radiate alternative polarizations in extended theories of gravity and may therefore contribute extra polarization modes to the stochastic gravitational-wave background [50,51]. Another potential source of stochastic backgrounds of alternative polarizations are the so-called “bubble walls” generated by first-order phase transitions in the early universe [52–54]. In scalar-tensor theories, bubbles are expected to produce strong monopolar emission [42]. Gravitational waves from bubbles are heavily redshifted, though, and today may have frequencies too low for Advanced LIGO to detect [53]. Bubble walls may therefore be a more promising target for future space-based detectors like LISA than for current ground-based instruments.

Finally, we note that it is also possible for alternative polarizations to be generated more efficiently from sources at very large distances. There are several ways in which this might occur. First, modifications to the gravitational-wave dispersion relation can lead to mixing between different polarizations in vacuum (an effect analogous to neutrino oscillations). This can cause mixing between the usual tensor modes [55], and also between tensor modes and other polarizations [56,57]. Thus, alternative polarizations can be generated during propagation, even if only tensor modes are produced at the source. This effect would build with the distance to a given gravitational-wave source. Such behavior is among the effects arising from generic Lorentz-violating theories of gravity [58,59]. While birefringence and dispersion of the standard plus and cross modes have been explored observationally in this context [59,60], the phenomenological implications of additional polarization modes remain an open issue at present. Second, in many alternative theories fundamental constants (such as Newton’s constant G) are elevated to dynamical fields; these fields may have behaved differently at earlier
stages in the universe’s evolution [61,62]. As a consequence, local constraints on scalar emission may not apply to emission from remote sources. Additionally, it is in principle possible for local sources to be affected by screening mechanisms that do not affect some remote sources [63].

### III. STOCHASTIC BACKGROUNDS OF ALTERNATIVE POLARIZATIONS

The stochastic background introduces a weak, correlated signal into networks of gravitational-wave detectors. Searches for the stochastic background therefore measure the cross-correlation,

$$ \hat{C}(f) \propto \tilde{s}_1^*(f)\tilde{s}_2(f), \quad (1) $$

between the strain $\tilde{s}_1(f)$ and $\tilde{s}_2(f)$ measured by pairs of detectors (see Ref. [17] for a comprehensive review of stochastic background detection methods).

We make several assumptions about the background. First, we assume that the stochastic background is isotropic, stationary, and Gaussian. Second, we assume that there are no correlations between different tensor, vector, and scalar polarization modes. We can therefore express the total measured cross power $\langle \hat{C}(f) \rangle$ as a sum of three terms due to each polarization sector. Finally, we assume that the tensor and vector sectors are individually unpolarized, with equal power in the tensor plus and cross modes and equal power in the vector-x and vector-y modes. This follows from the fact that we expect gravitational-wave sources to be isotropically distributed and randomly oriented with respect to Earth. In contrast, we cannot assume that the scalar sector is unpolarized. Scalar breathing and longitudinal modes cannot be rotated into one another via a coordinate transformation (as can the tensor plus and cross

modes, for instance), and so source isotropy does not imply equal power in each scalar polarization. However, the responses of the LIGO detectors to breathing and longitudinal modes are completely degenerate, and so Advanced LIGO is sensitive only to the total power in scalar modes rather than the individual energies in the breathing and longitudinal polarizations [12,30].

The above assumptions are not all equally justifiable, and may be broken by various alternative theories of gravity. For instance, one should not expect an unpolarized background in any theory that includes parity-odd gravitational couplings, like Chern-Simons gravity [64–67], even in the absence of nontensorial modes [68]. Furthermore, different polarizations may not be statistically independent, as is the case for the breathing and longitudinal modes in linearized massive gravity [69]. Finally, we should expect a departure from isotropy in any theory violating Lorentz invariance, like those within the standard model extension framework [55,58,59]. These exceptions notwithstanding, for simplicity we proceed under the assumptions listed above, leaving more generic cases for future work.

Under our assumptions, the measured cross power due to the background is given by [17,30,70]

$$ \langle \tilde{s}_1^*(f) \tilde{s}_2(f') \rangle = \delta(f - f') \gamma_a(f) H^a(f), \quad (2) $$

where repeated indices denote summation over tensor, vector, and scalar modes ($a \in \{T, V, S\}$). The overlap reduction functions $\gamma_a(f)$ quantify the sensitivity of detector pairs to isotropic backgrounds of each polarization [30,71] (see Appendix A for details). The functions $H^a(f)$, meanwhile, encode the spectral shape of the stochastic background within each polarization sector.

In the left-hand side of Fig. 2, we show the overlap reduction functions for the Hanford-Livingston (H1-L1)

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /><figcaption>FIG. 2. Overlap reduction functions quantifying the sensitivities of the Hanford-Livingston (left) and Hanford-Virgo (right) baselines to isotropic backgrounds of tensor-, vector-, and scalar-polarized gravitational waves. The distance between Hanford and Virgo is much larger than that between Hanford and Livingston; the Hanford-Virgo overlap reduction functions are therefore smaller in amplitude and more rapidly oscillatory.</figcaption></figure>
Advanced LIGO network. The overlap reduction functions are normalized such that $\gamma_T(f) = 1$ for coincident and coaligned detectors. For the Advanced LIGO network, the tensor overlap reduction function has magnitude $|\gamma_T(0)| = 0.89$ at $f = 0$, representing reduced sensitivity due to the separation and relative rotation of the H1 and L1 detectors. Additionally, the H1-L1 tensor overlap reduction function decays rapidly to zero above $f \approx 64$ Hz. Standard Advanced LIGO searches for the stochastic background therefore have negligible sensitivity at frequencies above ~64 Hz.

Relative to $\gamma_T(f)$, the H1-L1 vector overlap reduction function $\gamma_V(f)$ is of comparable magnitude at low frequencies, but remains non-negligible at frequencies above 64 Hz. As a result, we will see below that Advanced LIGO is in many cases more sensitive to vector-polarized backgrounds than standard tensor backgrounds. The scalar overlap reduction function, meanwhile, is smallest in magnitude, with $|\gamma_S(0)|$ a factor of 3 small than $|\gamma_T(0)|$ and $|\gamma_V(0)|$. Advanced LIGO is therefore least sensitive to scalar-polarized backgrounds. This reflects a generic feature of quadrupole gravitational-wave detectors, which geometrically have a smaller response to scalar modes than to vector and tensor polarizations [34]. For an extreme example of the opposite case, see pulsar timing arrays, which are orders of magnitude more sensitive to longitudinal polarizations than to standard tensor-polarized signals [72,73].

For comparison, the right-hand side of Fig. 2 shows the overlap reduction functions for the Hanford-Virgo (H1-V1) baseline. As the separation between Hanford and Virgo is much greater than that between Hanford and Livingston, the Hanford-Virgo overlap reduction functions are generally much smaller in amplitude and more rapidly oscillatory, translating into weaker sensitivity to the stochastic background. Note, however, that the H1-V1 tensor overlap reduction function remains larger in amplitude than H1-L1's at frequencies $f \gtrsim 200$ Hz, implying heightened relative sensitivity to tensor backgrounds at high frequencies [74].

The functions $H^a(f)$ appearing in Eq. (2) are theory independent; they are observable quantities that can be directly measured in the detector frame. Stochastic backgrounds are not conventionally described by $H(f)$, though, but by their gravitational-wave energy density [70],

$$ \Omega(f) = \frac{1}{\rho_c} \frac{d\rho_{GW}(f)}{d \ln f}, \qquad (3) $$

defined as the fraction of the critical energy density $\rho_c = 3H_0^2 c^2/(8\pi G)$ contained in gravitational waves per logarithmic frequency interval $d \ln f$. Here, $H_0$ is the Hubble constant and $G$ is Newton's constant. Within general relativity, the background's energy density is related to $H(f)$ via [70]

$$ \Omega(f) = \frac{20\pi^2}{3H_0^2} f^3 H(f). \qquad (4) $$

Equation (4) is a consequence of Isaacson's formula for the effective stress energy of gravitational waves [69,70,75]. Alternate theories of gravity, though, can predict different expressions for the stress energy of gravitational waves and, hence, different relationships between $H^a(f)$ and $\Omega^a(f)$ [69]. For ease of comparison to previous studies, we use Eq. (4) to define the canonical energy density $\Omega^a(f)$ in polarization sector a. If we allow Isaacson's formula to hold, then $\Omega^a(f)$ may be directly interpreted as a physical energy density. If not, though, then $\Omega^a(f)$ can instead be understood as a function of the observable $H^a(f)$.

We choose to normalize the cross-correlation statistic $\hat{C}(f)$ such that

$$ \langle \hat{C}(f) \rangle = \gamma_a(f) \Omega^a(f). \qquad (5) $$

Its variance is then [30,70]

$$ \sigma^2(f) = \frac{1}{2Tdf} \left( \frac{10\pi^2}{3H_0^2} \right)^2 f^6 P_1(f) P_2(f). \qquad (6) $$

Here, $T$ is the total coincident observation time between detectors, $df$ is the frequency bin width considered, and $P_i(f)$ is the noise power spectral density of detector $i$. Note that the normalization of our cross-correlation measurement, with the overlap reduction functions appearing in $\langle \hat{C}(f) \rangle$ rather than $\sigma^2(f)$, differs from the convention normally adopted in the literature. Standard stochastic searches typically define a statistic $\hat{Y}(f) \propto \tilde{s}_1^*(f)\tilde{s}_2(f)/\gamma_T(f)$, such that $\langle \hat{Y}(f) \rangle = \Omega^T(f)$ in the presence of a pure tensor background [25,76,77]. Our choice of normalization, though, will prove more convenient when studying stochastic backgrounds of mixed gravitational-wave polarizations. To emphasize this distinction, we denote our cross-power estimators by $\hat{C}(f)$, rather than the more common $\hat{Y}(f)$.

A spectrum of cross-correlation measurements $\hat{C}(f)$ may be combined to obtain a single broadband signal-to-noise ratio (SNR), given by

$$ \text{SNR}^2 = \frac{(\hat{C}|\gamma_a \Omega_M^a)^2}{(\gamma_b \Omega_M^b | \gamma_c \Omega_M^c)}, \qquad (7) $$

where we define the inner product

$$ (A|B) = \left( \frac{3H_0^2}{10\pi^2} \right)^2 2T \int_0^\infty \frac{\tilde{A}^*(f)\tilde{B}(f)}{f^6 P_1(f) P_2(f)} df. \qquad (8) $$

In Eq. (7), $\Omega_M^a(f)$ is our adopted model for the energy-density spectrum of the stochastic background. The expected SNR is maximized when this model is equal to
<figure><img src="image_5.png" /><figcaption>FIG. 3. Left: PI curves showing the sensitivity of Advanced LIGO to stochastic backgrounds of tensor, vector, and scalar polarizations (solid blue, red, and green curves, respectively). Power-law energy-density spectra [Eq. (10)] drawn tangent to the PI curves have expected $\langle \text{SNR}_{\text{OPT}} \rangle = 3$ after 3 years of observation at design sensitivity. Also shown are “naive” PI curves for vector and scalar backgrounds (dashed red and green curves) illustrating the sensitivity of existing search methods optimized only for tensor polarizations. Right: Minimum detectable background amplitudes ($\langle \text{SNR}_{\text{OPT}} \rangle = 3$ after 3 years of observation at design sensitivity) as a function of spectral index $\alpha_a$. For small and negative values of $\alpha_a$, Advanced LIGO is approximately equally sensitive to tensor- and vector-polarized backgrounds. For large $\alpha_a$, Advanced LIGO is instead most sensitive to vector- and scalar-polarized backgrounds. The dashed curves show amplitudes detectable with existing “naive” methods. The sensitivity loss between optimal and naive cases is negligible for $\alpha_a \lesssim 0$, but becomes significant at moderate positive slopes (e.g., $\alpha_a \sim 2$). The kinks in the naive curves are due to biases incurred when recovering vector and scalar backgrounds with purely tensor models; see the text for details.</figcaption></figure>

the background’s true energy-density spectrum. The resulting optimal SNR is given by

$$ \text{SNR}_{\text{OPT}}^2 = (\gamma_a \Omega^a | \gamma_b \Omega^b) \qquad (9) $$

(see Appendix B for details).

Conventionally, stochastic energy-density spectra are modeled as power laws, such that

$$ \Omega_M^a(f) = \Omega_0^a \left( \frac{f}{f_0} \right)^{\alpha_a}, \qquad (10) $$

where $\Omega_0^a$ is the background’s amplitude at a reference frequency $f_0$ and $\alpha_a$ is its spectral index (or slope) [25,70,77]. The predicted tensor stochastic background from compact binary coalescences, for instance, is well modeled by a power law of slope $\alpha_T = 2/3$ in the sensitivity band of Advanced LIGO [76]. For reference, slopes of $\alpha = 0$ and $\alpha = 3$ correspond to scale-invariant energy and strain spectra, respectively. While we largely assume power-law models in our analysis, in Sec. VI we explore the potential consequences if this assumption is in fact incorrect (as would be the case, for instance, for a background of unexpectedly massive binary black holes [76]). Throughout this paper we use the reference frequency $f_0 = 25$ Hz.

With the above formalism in hand, we can quantify Advanced LIGO’s sensitivity to stochastic backgrounds of alternative polarizations. Plotted on the left-hand side of Fig. 3 are power-law integrated (PI) curves representing Advanced LIGO’s optimal sensitivity to power-law

backgrounds of pure tensor (solid blue), vector (solid red), and scalar (solid green) modes [78]. The PI curves are defined such that a power-law spectrum drawn tangent to the PI curve will be marginally detectable with $\langle \text{SNR}_{\text{OPT}} \rangle = 3$ after 3 years of observation with design-sensitivity Advanced LIGO. In general, energy-density spectra lying above and below the PI curves are expected to have optimal SNRs greater and less than 3, respectively. In the right-hand side of Fig. 3, meanwhile, the solid curves trace the power-law amplitudes required for marginal detection ($\langle \text{SNR}_{\text{OPT}} \rangle = 3$ after 3 years of observation) as a function of spectral index. Incidentally, the left- and right-hand subplots of Fig. 3 are Legendre transforms of one another.

For spectral indices $\alpha_a \lesssim 0$, Advanced LIGO is approximately equally sensitive to tensor- and vector-polarized backgrounds, with reduced sensitivity to scalar signals. When $\alpha_a = 0$, for instance, the minimum optimally detectable tensor and vector amplitudes are $\Omega_0^T = 1.1 \times 10^{-9}$ and $\Omega_0^V = 1.5 \times 10^{-9}$, while the minimum detectable scalar amplitude is $\Omega_0^S = 4.4 \times 10^{-9}$, a factor of 3 larger. This relative sensitivity is due to the fact that the tensor and vector overlap reduction functions are of comparable magnitude at low frequencies, while the scalar overlap reduction function is reduced in size (see Fig. 2).

At high frequencies, on the other hand, Advanced LIGO’s tensor overlap reduction function decays more rapidly than the vector and scalar overlap reduction functions. As a result, Advanced LIGO is more sensitive to vector and scalar backgrounds of large, positive slope
than to tensor backgrounds of similar spectral shape. In Fig. 3, for instance, the vector and scalar PI curves are seen to lie an order of magnitude below the tensor PI curve at frequencies above $f \sim 300$ Hz. The constraints that Advanced LIGO can place on positively sloped vector and scalar backgrounds are therefore as much as an order of magnitude more stringent than those that can be placed on tensor backgrounds of similar slope.

We emphasize that the Advanced LIGO network's relative sensitivities to tensor-, vector-, and scalar-polarized backgrounds are due purely to its geometry, rather than properties of the backgrounds themselves. If we were instead to consider the Hanford-Virgo baseline, for instance, the right-hand side of Fig. 2 shows that at high frequencies the H1-V1 pair is least sensitive to scalar polarizations, whereas the H1-L1 baseline is least sensitive to tensor modes.

So far we have discussed only Advanced LIGO's *optimal* sensitivity to stochastic backgrounds of alternative polarizations. Existing stochastic searches, though, are *not* optimized for such backgrounds, instead using models $\Omega_M^a(f)$ that allow only for tensor gravitational-wave polarizations. The dashed curves in Fig. 3 illustrate Advanced LIGO's “naive” sensitivity to backgrounds of alternative polarizations when incorrectly assuming a purely tensor power-law model. Note that the naive curves on the right-hand side of Fig. 3 are not smooth, with sharp kinks at $\alpha_a \sim 2$; more on this below. The loss in sensitivity between the optimal and naive searches varies greatly with different spectral indices. Sensitivity loss is relatively minimal for slopes $\alpha_a \lesssim 0$. When $\alpha_S = 0$, for example, the minimum detectable scalar amplitude rises from $\Omega_0^S = 4.4 \times 10^{-9}$ in the optimal case to $5.3 \times 10^{-9}$ in the naive case, an increase of 20%. Thus, a flat scalar background that is optimally detectable by Advanced LIGO may still be detected using existing techniques tailored to tensor polarizations. The SNR penalty is more severe for stochastic backgrounds of moderate positive slope. For $\alpha_S = 2$, Advanced LIGO can optimally detect a scalar background of amplitude $\Omega_0^S = 1.3 \times 10^{-9}$, while existing methods would detect only a background of amplitude $\Omega_0^S = 4.4 \times 10^{-9}$, a factor of 3.4 larger.

Since the SNR of the stochastic search accumulates only as SNR $\propto \sqrt{T}$, even a small decrease in sensitivity can result in a somewhat severe increase in the time required to make a detection. To illustrate this, Fig. 4 shows the ratio $T_{naive}/T_{optimal}$ between the observing times required for Advanced LIGO to detect vector (red curve) and scalar (green curve) backgrounds using existing naive methods and optimal methods. Although we noted above that existing methods incur little sensitivity loss to flat scalar backgrounds, the detection of such backgrounds would nevertheless require at least 50% more observing time with existing searches. Since the stochastic background is expected to be optimally detected only after several years,

<figure><img src="image_6.png" /><figcaption>FIG. 4. The fractional increase in observing time required for Advanced LIGO to make a detection of vector (red curve) and scalar (green curve) backgrounds using existing search techniques, as a function of their spectral index $\alpha_a$. The sharp kinks in each curve are due to biases incurred when fitting vector and scalar backgrounds with a purely tensor model; see the text for details.</figcaption></figure>

even a 50% increase potentially translates into years of additional observation time, a requirement that may well stress standard experimental lifetimes and operational funding cycles. Naive detection of a scalar background with $\alpha_S = 2$, for comparison, would require nearly 12 times the observing time.

Figures 3 and 4 both show conspicuous kinks occurring at $\alpha_S \approx 1.75$ and $\alpha_V \approx 2.5$. These features are due to severe systematic parameter biases incurred when recovering vector and scalar backgrounds with a purely tensorial model. For vector and scalar backgrounds of with $\alpha_a \gtrsim 3$, the best-fit slope $\alpha_T$ (which maximizes the recovered SNR) is biased towards large values. Meanwhile, vector and scalar backgrounds with $\alpha_a \lesssim 1$ bias $\alpha_T$ in the opposite direction, towards smaller values. The sharp kinks in Figs. 3 and 4 occur at the transition between these two regimes. Such biases indicate another pitfall of existing search methods designed only for tensor polarizations. Even if a vector- or scalar-polarized background is recovered with minimal SNR loss, without some independent confirmation we may remain entirely unaware that the detected background indeed violates general relativity (see Sec. IV). Furthermore, we would suffer from severe “stealth bias,” unknowingly recovering heavily biased estimates of the amplitude and spectral index of the stochastic background [79,80].

#### IV. IDENTIFYING ALTERNATIVE POLARIZATIONS

We have seen in Sec. III that, even when using existing methods assuming only standard tensor polarizations, Advanced LIGO may still be capable of detecting a stochastic background of vector or scalar modes (albeit
<figure><img src="image_7.png" /><figcaption>FIG. 5. Left: Simulated cross-correlation measurements $\hat{C}(f)$ for purely tensor (blue) and purely scalar (green) stochastic backgrounds, recovered after 3 years of observation with design-sensitivity Advanced LIGO. The backgrounds shown have $\alpha_T = \alpha_S = 2/3$, and have amplitudes chosen such that each is detectable with $\langle \text{SNR}_{\text{OPT}} \rangle = 150$. The measured spectra each show distinct modulations characteristic of the tensor and scalar overlap reduction functions, allowing a clear identification of the polarization in each case. Right: Simulated recovery of weaker tensor and scalar backgrounds, detectable with $\langle \text{SNR}_{\text{OPT}} \rangle = 5$ after 3 years of observation at design sensitivity. While each background would be confidently detected by existing search techniques, the characteristic amplitude modulations and hence the polarization content of each simulated background are no longer evident.</figcaption></figure>

after potentially much longer observation times). Detection is only the first of two hurdles, though. Once the stochastic background has been detected, we still need to establish whether it is entirely tensor polarized or if it contains vector- or scalar-polarized gravitational waves.

Since tensor, vector, and scalar gravitational-wave polarizations each enter into cross-correlation measurements [Eq. (2)] with unique overlap reduction functions, the polarization content of a detected stochastic background is in principle discernible from the spectral shape of $\hat{C}(f)$. As an example, Fig. 5 shows simulated cross-correlation measurements $\hat{C}(f)$ for both purely tensor-polarized (blue) and purely scalar-polarized (green) backgrounds after 3 years of observation with design-sensitivity Advanced LIGO. The left-hand side shows simulated measurements of extremely strong backgrounds, with spectra $\Omega^T(f) = 5 \times 10^{-8} (f/f_0)^{2/3}$ and $\Omega^S(f) = 1.8 \times 10^{-7} (f/f_0)^{2/3}$; these amplitudes are chosen such that each background has expected $\langle \text{SNR}_{\text{OPT}} \rangle = 150$ after 3 years of observation. The dashed curves trace the expectation values $\langle \hat{C}(f) \rangle$ of the cross-correlation spectra for each case, while the solid curves show a particular instantiation of measured values. The alternating signs (positive or negative) of each spectrum are determined by the tensor and scalar overlap reduction functions, which have zero crossings at different characteristic frequencies (see Fig. 2). As a result, tensor- and scalar-polarized signals each impart a unique shape to the cross-correlation spectra, offering a means of discriminating between the two cases.

As mentioned above, though, the backgrounds shown on the left-hand side of Fig 5 are unphysically loud, with $\text{SNR}_{\text{OPT}} = 152$ and 148 for the simulated tensor and scalar backgrounds, respectively. A tensor background of this

amplitude would have been detectable with the standard isotropic search over Advanced LIGO’s O1 observing run [25]. Since stochastic searches accumulate SNR over time, the first detection of the stochastic background will necessarily be marginal; in this case, the presence of alternative gravitational-wave polarizations would not be clear. To demonstrate this, the right-hand side of Fig. 5 shows the simulated recovery of weaker tensor and scalar backgrounds of spectral shape $\Omega^T(f) = 1.7 \times 10^{-9} (f/f_0)^{2/3}$ and $\Omega^S(f) = 6.1 \times 10^{-9} (f/f_0)^{2/3}$, again after 3 years of observation with Advanced LIGO. These amplitudes correspond to expected $\langle \text{SNR}_{\text{OPT}} \rangle = 5$ after 3 years. While Advanced LIGO would still make a very confident detection of each background, with $\text{SNR}_{\text{OPT}} = 6.7$ and 7.8 for the simulated tensor and scalar cases, the backgrounds’ polarization content is no longer obvious.

Interestingly, even when naively searching for purely tensor polarized backgrounds, design-sensitivity Advanced LIGO still detects the “quiet” scalar example (on the right-hand side of Fig. 5) with SNR = 5.0. When assuming a priori that the stochastic background is purely tensor polarized, any vector or scalar components detected with existing techniques may therefore be mistaken for ordinary tensor modes. Not only would vector or scalar components fail to be identified, but, as discussed in Sec. III, they would heavily bias parameter estimation of the tensor energy-density spectrum. If we wish to test general relativity with the stochastic background, we will therefore need to develop new tools in order to formally quantify the presence (or absence) of vector or scalar polarizations. Additionally, while we have so far investigated only backgrounds of pure tensor, vector, or scalar polarization, most plausible alternative theories of gravity will predict
backgrounds of mixed polarization, with vector or scalar
components in addition to a tensor component. Any
realistic approach must therefore be able to handle a
stochastic background of completely generic polarization
content.

Our approach is to detect and classify the stochastic
background using Bayesian model selection, adapting the
method used in Ref. [34] to study the polarization content
of continuous gravitational-wave sources. First, we define
an odds ratio $O_{\mathrm{N}}^{\mathrm{SIG}}$ between signal (SIG) and noise (N)
hypotheses to determine if a stochastic background (of any
polarization) has been observed. Once a background is
detected, we then construct a second odds ratio $O_{\mathrm{GR}}^{\mathrm{NGR}}$
to determine if the background contains only tensor
polarization consistent with general relativity (GR hypoth-
esis) or if there exist alternative polarizations, inconsistent
with general relativity (the NGR hypothesis). We describe
the definitions of these hypotheses and the construction of
$O_{\mathrm{N}}^{\mathrm{SIG}}$ and $O_{\mathrm{GR}}^{\mathrm{NGR}}$ in detail in Appendix C. Unlike existing
detection methods that assume a pure tensor background,
our scheme allows for the detection of generically polarized
stochastic backgrounds. It encapsulates the optimal detec-
tion of tensor, vector, and scalar polarizations as described
in Sec. III, and moreover enables the detection of more
complex backgrounds of mixed polarization.

To compute the odds ratios $O_N^{\text{SIG}}$ and $O_{\text{GR}}^{\text{NGR}}$, we use the PyMultiNest package [81], which implements a PYTHON wrapper for the nested sampling software MultiNest [82–84]. MultiNest, an implementation of the nested sampling algorithm [85,86], is designed to efficiently evaluate Bayesian evidences [see Eq. (C1)] in high-dimensional parameter spaces, even in the case of large and possibly curving parameter degeneracies. At little additional computational cost, MultiNest also returns posterior probabilities for each model parameter, allowing for parameter estimation in addition to model selection. Details associated with running MultiNest are given in Appendix D.

Our approach fundamentally differs from the strategy
proposed by Nishizawa *et al.* in Refs. [30–32]. Nishizawa
*et al.* endeavor to separate and measure the background’s
tensor, vector, and scalar content within each frequency bin.
To solve for these three unknowns, three pairs of gravita-
tional-wave detectors are required to break the degeneracy
between polarizations. A nice feature of their method is that
it allows for the separation of polarization modes without
the need for a parametrized model of the background’s
energy-density spectrum. However, it has several draw-
backs. First, the Nishizawa *et al.* component separation
scheme requires at least three detectors. Even then, this
method is not very sensitive; covariances between polari-
zation modes mean that only very loud backgrounds can be
separated and independently detected with reasonable
confidence. Finally, Nishizawa *et al.* largely focus on the
detection of a background, not the characterization of its

spectral shape. Reference [32] does discuss parameter
estimation on the stochastic background using a Fisher
matrix formalism, but this approach has well-known
limitations [87].

Our method is more aggressive. Rather than attemp-
ting to resolve the relative polarization content within
each frequency bin, we assume a power-law model for
the energy density in each polarization mode (see
Appendix C). This allows us to confidently detect far
weaker signals than the approach of Nishizawa et al. While
this approach is potentially susceptible to bias if our model
poorly fits the true background, it is a reasonable model for
astrophysically plausible scenarios. Even if the true back-
ground differs significantly from this model, we find in
Sec. VI that potential bias is negligible. Another advantage
of our method is that it can be used with only two detectors
and, hence, can be applied today, rather than waiting for the
construction of future gravitational-wave detectors. Finally,
in Sec. V, we show that our Bayesian approach allows for
full parameter estimation on the stochastic background,
which properly takes into account the full degeneracies
between background parameters (something a Fisher
matrix analysis cannot do).

A. Backgrounds of single polarizations

As a first demonstration of this machinery, we explore
the simple cases of purely tensor-, vector-, or scalar-
polarized stochastic backgrounds. Shown in Fig. 6 are
distributions of odds ratios $O_N^{\text{SIG}}$ and $O_{\text{GR}}^{\text{NGR}}$ obtained for
simulated observations of both tensor and scalar back-
grounds, each of slope $\alpha = 2/3$ (the characteristic slope of
a tensor binary black hole background). For each polari-
zation, we consider two choices of amplitude, correspond-
ing to $\langle \text{SNR}_{\text{OPT}} \rangle = 5$ and 10 after 3 years of observation
with design-sensitivity Advanced LIGO. For comparison,
the hatched gray distributions show odds ratios obtained in
the presence of pure Gaussian noise.

As seen in the left-hand side of Fig. 6, Gaussian noise
yields a narrow odds ratio distribution centered at
$\ln O_N^{\text{SIG}} \approx -1.0$. In contrast, the simulated observations of
tensor and scalar backgrounds yield large, positive odds
ratios, well separated from Gaussian noise. Note that the
tensor and scalar distributions lie nearly on top of one
another, as $O_N^{\text{SIG}}$ depends primarily on the optimal SNR of a
background and not its polarization content.

The right-hand side of Fig. 6, in turn, shows the
odds ratios $O_{\text{GR}}^{\text{NGR}}$ quantifying the evidence for alternative
polarization modes. In the case of pure Gaussian noise,
we again see a narrow distribution of odds ratios, centered
at $\ln O_{\text{GR}}^{\text{NGR}} \approx -0.4$. In the absence of informative data,
our analysis thus slightly favors the GR hypothesis. This
can be understood as a consequence of the implicit
Bayesian “Occam’s factor,” which penalizes the more
complex NGR hypothesis over the simpler GR hypothesis.
Simulated observations of scalar backgrounds, in turn,
<figure><img src="image_8.png" /><figcaption>FIG. 6. Left: Distributions of odds ratios $O_N^{\text{SIG}}$ between signal and noise hypotheses for simulated observations of tensor (blue) and scalar (green) stochastic backgrounds of slope $\alpha = 2/3$, assuming 3 years of observation with design-sensitivity Advanced LIGO. We consider two different strengths for each polarization, corresponding to $\langle \text{SNR}_{\text{OPT}} \rangle = 5$ and 10. For each background strength, the tensor and scalar odds ratios lie nearly on top of one another. Also shown is the background distribution of odds ratios obtained when observing pure Gaussian noise (hatched gray). In the presence of a stochastic background, the recovered odds ratios grow as $\ln O_N^{\text{SIG}} \propto \text{SNR}_{\text{OPT}}^2$, showing increasingly large preference for the signal hypothesis. Right: Odds ratios $O_{\text{GR}}^{\text{NGR}}$ between NGR and GR hypotheses obtained for the same set of simulated Advanced LIGO observations. In the presence of a tensor-polarized background, we recover narrow distributions of odds ratios centered at $\ln O_{\text{GR}}^{\text{NGR}} \approx -1.4$, reflecting consistency with the GR hypothesis. A scalar background, on the other hand, yields large positive odds ratios, correctly showing a strong preference for our NGR hypothesis.</figcaption></figure>

yield large positive values for $\ln O_{\text{GR}}^{\text{NGR}}$, correctly preferencing the NGR hypothesis. In contrast, pure tensor backgrounds yield negative $\ln O_{\text{GR}}^{\text{NGR}}$. Interestingly, the recovered odds ratios do not grow increasingly negative with larger tensor amplitudes, but instead saturate at $\ln O_{\text{GR}}^{\text{NGR}} \approx -1.4$. This reflects the fact that a nondetection of vector or scalar polarizations can never strictly rule out their presence, but only place an upper limit on their amplitudes. In other words, a strong detection of a pure tensor stochastic background cannot provide evidence for the GR hypothesis, but at best only offers no evidence against it. This behavior is in part due to our choice of amplitude priors, which allow for finite but immeasurably small vector and scalar energy densities (see Appendix C).

Figure 7 illustrates more generally how $O_N^{\text{SIG}}$ (left-hand column) and $O_{\text{GR}}^{\text{NGR}}$ (right-hand column) scale with the amplitudes of purely tensor (blue), vector (red), and scalar (green) stochastic backgrounds. Black points mark odds ratios computed from individual realizations of simulated data, while the solid curves and shaded regions trace their smoothed mean and standard deviation. We again see $\ln O_N^{\text{SIG}}$ increasing monotonically with injected amplitude for all three polarizations. Specifically, $O_N^{\text{SIG}}$ depends inversely on the noise-hypothesis likelihood [defined by Eq. (C5)] and therefore scales as

$$ \ln O_N^{\text{SIG}} \propto \text{SNR}_{\text{OPT}}^2. \qquad (11) $$

As seen earlier in Fig. 6, $\ln O_{\text{GR}}^{\text{NGR}}$ saturates at $-1.4$ for loud tensor backgrounds. In the case of vector and scalar backgrounds, on the other hand, $\ln O_{\text{GR}}^{\text{NGR}}$ grows

quadratically with increasing amplitude. In particular, $\ln O_{\text{GR}}^{\text{NGR}}$ is proportional to the squared SNR of the residuals between the observed $\hat{C}(f)$ and the best-fit tensor model. We begin to see a strong preference for the NGR hypothesis when these residuals become statistically significant.

## B. Backgrounds of mixed polarization

So far, we have considered only cases of pure tensor, vector, or scalar polarization. Plausible alternative theories of gravity, however, would typically predict a mixed background of multiple polarization modes. How does our Bayesian machinery handle a background of mixed polarization? To answer this question, we investigate backgrounds of mixed tensor and scalar polarization. Figure 8 shows values of $O_N^{\text{SIG}}$ and $O_{\text{GR}}^{\text{NGR}}$ (left- and right-hand sides, respectively) as a function of the amplitude of each polarization. While we allow the amplitudes to vary, we fix the tensor and scalar slopes to $\alpha_T = 2/3$ (as predicted for binary black hole backgrounds) and $\alpha_S = 0$.

In the left-hand side Fig. 8, the recovered values of $\ln O_N^{\text{SIG}}$ simply trace contours of total energy. Thus, the detectability of a mixed background depends only on its total measured energy, rather than its polarization content. Meanwhile, three distinct regions are observed in the right-hand subplot. First, for small tensor and scalar amplitudes ($\log \Omega_0^T \lesssim -9.0$ and $\log \Omega_0^S \lesssim -8.5$), we obtain $\ln O_{\text{GR}}^{\text{NGR}} \approx -0.4$. In this region, the mixed background simply cannot be detected and so we recover the slight Occam's bias towards the GR hypothesis, as noted above. Second, for small scalar and large tensor amplitudes
<figure><img src="image_9.png" /><figcaption>FIG. 7. Odds ratios $O_N^{\text{SIG}}$ (left) and $O_{\text{GR}}^{\text{NGR}}$ (right) for simulated Advanced LIGO observations of purely tensor- (blue), vector- (red), and scalar-polarized (green) stochastic backgrounds. Within each plot, we show 750 simulated observations, with random log amplitudes chosen uniformly over the range $-10 < \log \Omega_0 < -7$. Black points mark the results from individual realizations, while the solid curves and shaded regions show the moving mean and standard deviations (smoothed with a Gaussian kernel) of these realizations. For each polarization, $\ln O_N^{\text{SIG}}$ scales quadratically with the amplitude of the stochastic background. Similarly, $\ln O_{\text{GR}}^{\text{NGR}}$ scales quadratically with vector and scalar amplitude. For tensor backgrounds, however, $\ln O_{\text{GR}}^{\text{NGR}}$ instead saturates at approximately $-1.4$.</figcaption></figure>

($\log \Omega_0^T \gtrsim -9.0$), the recovered odds ratios decrease to $\ln O_{\text{GR}}^{\text{NGR}} \approx -1.4$. This corresponds to the detection of the tensor component alone; the decrease in odds ratios is the same behavior previously seen in Figs. 6 and 7. Finally,

when $\Omega_0^S$ is large, the scalar component is detectable and the recovered $\ln O_{\text{GR}}^{\text{NGR}}$ increases rapidly to large, positive values. The threshold value of $\Omega_0^S$ at which $\ln O_{\text{GR}}^{\text{NGR}}$ becomes positive shows only little dependence on the
<figure><img src="image_10.png" /><figcaption>FIG. 8. Odds ratios for simulated Advanced LIGO measurements of stochastic backgrounds containing both tensor and scalar polarizations, assuming 3 years of observation at design sensitivity. The tensor and scalar components have slopes $\alpha_T = 2/3$ and $\alpha_S = 0$, respectively. Left: Odds ratios between signal and noise hypotheses. The observed values of $O_N^{\text{SIG}}$ trace contours in total background energy. Thus, the detectability of a background depends largely on its total power, not its polarization content. Right: Odds ratios $O_{\text{GR}}^{\text{NGR}}$ between NGR and GR hypotheses. Advanced LIGO would confidently identify the presence of the scalar background component when $\log \Omega_0^S \gtrsim -7.9$. LIGO's sensitivity to the scalar component is nearly independent of the strength of the tensor component; the minimum identifiable scalar amplitude $\Omega_0^S$ rises only slightly with increasing $\Omega_0^T$.</figcaption></figure>

amplitude of any tensor background which might also be present. When $\Omega_0^T$ is small, for instance, scalar amplitudes of size $\log \Omega_0^S \gtrsim -7.9$ are required to significantly prefer-ence the NGR model. When $\Omega_0^T$ is large, this requirement increases only slightly to $\log \Omega_0^S \gtrsim -7.8$. Thus, we should expect Advanced LIGO to be able to both detect and identify as nontensorial a flat scalar background of ampli-tude $\log \Omega_0^S \gtrsim -8$, regardless of the presence of an addi-tional tensor component.

It should be pointed out that positive $\ln O_{\text{GR}}^{\text{NGR}}$ indicates only that there exists evidence for alternative polarizations. From the odds ratio alone we cannot infer which specific polarizations—vector and/or scalar—are present in the background. While we found above that Advanced LIGO can identify mixed tensor-scalar backgrounds as nontensorial when $\log \Omega_0^S \gtrsim -8$, this does not imply that we can successfully identify the scalar component as such, only that our measurements are not consistent with tensor polarization alone (see Sec. V).

The future addition of new gravitational wave detectors will extend the reach of stochastic searches and help to break degeneracies between backgrounds of different polarizations. This expansion recently began with the completion of Advanced Virgo, which joined Advanced LIGO during its O2 observing run in August 2017 [2,7]. It is therefore interesting to investigate how the introduction of Advanced Virgo will improve the above results. Given detectors indexed by $i \in \{1, 2, ...\}$, the total SNR of a stochastic background is the quadrature sum of SNRs from each detector pair [70]:

$$ \text{SNR}^2 = \sum_{i} \sum_{j>i} \text{SNR}_{ij}^2, \qquad (12) $$

where each $\text{SNR}_{ij}$ is computed following Eq. (7). Naively, the SNR with which a background is observed is expected to increase as SNR $\propto \sqrt{N}$, where N is the total number of available detector pairs (three in the case of the Advanced LIGO-Virgo network). However, both the Hanford-Virgo and Livingston-Virgo pairs exhibit reduced sensitivity to the stochastic background due to their large physical separations. This fact is reflected in their respective overlap reduction functions, which are a factor of several smaller in magnitude than the Hanford-Livingston overlap reduction functions (see Fig. 2).

Given three independent detector pairs (and hence three independent measurements at each frequency), one can in principle directly solve for the unknown tensor, vector, and scalar contributions to the background in each frequency bin [17,30–32]. This component separation scheme can be performed without resorting to a model for the stochastic energy-density spectrum. However, frequency-by-frequency component separation is unlikely to be successful using the LIGO-Virgo network, due to the large uncertainties in the measured background at each frequency. Instead, when considering joint Advanced LIGO-Virgo observations, we again apply the Bayesian framework introduced above, leveraging measurements made at many frequencies in order to constrain the power-law amplitude and slope of each polarization mode.

To quantify the extent to which Advanced Virgo aids in the detection of the stochastic background, we again consider simulated observations of a mixed tensor (slope $\alpha_T = 2/3$) and scalar (slope $\alpha_S = 0$) background, this time with a three-detector Advanced LIGO-Virgo network. Our Bayesian formalism is easily extended to accommodate the case of multiple detector pairs; details are given in Appendix C. The odds ratios obtained from our simulated
<figure><img src="image_11.png" /><figcaption>FIG. 9. As above, but for simulated 3-year observations with the joint Advanced LIGO-Virgo network at design sensitivity. Despite the inclusion of Advanced Virgo, the sensitivity of this three-detector network is nearly identical to that of Advanced LIGO alone.</figcaption></figure>

Advanced LIGO-Virgo observations are shown in Fig. 9 for various tensor and scalar amplitudes. The inclusion of Advanced Virgo yields no clear improvement over the Advanced LIGO results in Fig. 8. Because of its large distance from LIGO, Advanced Virgo does not contribute more than a small fraction of the total observed SNR. As a result, the combined Hanford-Livingston-Virgo network both detects (as indicated with $O_N^{SIG}$) and identifies (via $O_{GR}^{NGR}$) the scalar background component with virtually the same sensitivity as the Hanford-Livingston network alone.

## V. PARAMETER ESTIMATION ON MIXED BACKGROUNDS

Parameter estimation will be the final step in a search for a stochastic background of generic polarization. If a gravitational-wave background is detected (as inferred from $O_N^{SIG}$), how well can Advanced LIGO constrain the properties of the background? Alternatively, if no detection is made, what upper limits can Advanced LIGO place on the background amplitudes of each polarization mode? We investigate these questions through three case studies: an observation of pure Gaussian noise, a standard tensor stochastic background, and a background of mixed tensor and scalar polarizations. The simulated background parameters used for each case are listed in Table I.

When performing model selection, the odds ratios $O_N^{SIG}$ and $O_{GR}^{NGR}$ are constructed by independently allowing for

each combination of tensor, vector, and scalar modes (see Appendix C). Parameter estimation, meanwhile, must be performed in the context of a specific background model. For the case studies below, we adopt the broadest possible hypothesis, allowing for the existence of tensor, vector, and scalar polarizations; we call this our “TVS” hypothesis (see additional details in Appendix C). This choice will enable us to place simultaneous constraints on the presence of all three polarization modes. Within our TVS hypothesis, we model the stochastic energy-density spectrum as a sum of three power-laws, one from each polarization sector: $\Omega_{TVS}(f) = \Omega_0^T(f/f_0)^{\alpha_T} + \Omega_0^V(f/f_0)^{\alpha_V} + \Omega_0^S(f/f_0)^{\alpha_S}$. Posterior distributions on the amplitudes $\Omega_0^a$ and spectral indices $\alpha_a$ are obtained using MultiNest, which returns samples drawn from the measured posterior distributions.

There are several key subtleties that must be understood when interpreting the parameter estimation results presented below. First, whereas standard tensor upper limits are conventionally defined with respect to a single, fixed slope [25,77], we quote amplitude limits obtained *after* marginalization over spectral index. This approach concisely combines information from the entire posterior parameter space to offer a single limit on each polarization considered. As a result, however, our simulated upper limits presented here should not be directly compared to those from standard searches for tensor backgrounds. Second, parameter estimation results are contingent upon the choice of a specific model. While we demonstrate parameter estimation results

<table><caption>TABLE I. Stochastic background parameters used for each case study presented. For each case, the vector amplitude is set to zero. Also shown are the odds ratios computed for each simulated observation.</caption><thead><tr><th rowspan="2">Case</th><th rowspan="2">log Ω<sup>T</sup><sub>0</sub></th><th rowspan="2">α<sub>T</sub></th><th rowspan="2">log Ω<sup>S</sup><sub>0</sub></th><th rowspan="2">α<sub>S</sub></th><th colspan="2">H1-L1</th><th colspan="2">H1-L1-V1</th></tr><tr><th>ln O<sup>SIG</sup><sub>N</sub></th><th>ln O<sup>NGR</sup><sub>GR</sub></th><th>ln O<sup>SIG</sup><sub>N</sub></th><th>ln O<sup>NGR</sup><sub>GR</sub></th></tr></thead><tbody><tr><td>1. Noise</td><td>...</td><td>...</td><td>...</td><td>...</td><td>-1.1</td><td>-0.4</td><td>-1.1</td><td>-0.4</td></tr><tr><td>2. Tensor</td><td>-8.78</td><td>0.67</td><td>...</td><td>...</td><td>8.4</td><td>-1.4</td><td>8.8</td><td>-1.4</td></tr><tr><td>3. Tensor + scalar</td><td>-8.48</td><td>0.67</td><td>-7.83</td><td>0.0</td><td>193.5</td><td>16.1</td><td>197.3</td><td>19.3</td></tr></tbody></table>
under our TVS hypothesis (see Appendix C), other hypotheses
may be better suited to answering other experimental ques-
tions. For example, if we were specifically interested in
constraining scalar-tensor theories (which a priori do not
allow vector polarizations), we would instead perform param-
eter estimation under a hypothesis containing only tensor and
scalar modes. And if our goal was to perform a standard
stochastic search for a purely tensor-polarized background, we
would restrict to a hypothesis admitting only tensor polar-
izations. Although these various hypotheses all contain an
analogous parameter $\Omega_0^T$, the resulting upper limits on $\Omega_0^T$ will
generically be different in each case. In short, different
experimental questions will yield different answers.

*Case 1: Gaussian noise.*—First, we consider the case of pure noise, producing a simulated 3-year observation of Gaussian noise at Advanced LIGO’s design sensitivity. The resulting posteriors are shown in Fig. 10. The colored histograms along the diagonal show the marginalized 1D posteriors for the amplitudes and slopes of the tensor, vector, and scalar components (blue, green, and red, respectively). The priors placed on each parameter are indicated with a dashed gray curve. Above each posterior we quote the median posterior value as well as ±34% credible limits. The remaining subplots illustrate the joint 2D posteriors between each pair of parameters.

For this simulated Advanced LIGO observation, we obtain $\ln \mathcal{O}_N^{\text{SIG}} = -1.1$, consistent with a null detection. Accordingly, the posteriors on $\Omega_0^T$, $\Omega_0^V$, and $\Omega_0^S$ are each consistent with the lower bound of our amplitude prior (at $\log \Omega_{\text{min}} = -13$). Meanwhile, the posteriors on spectral indices $\alpha_T$, $\alpha_V$, and $\alpha_S$ simply recover our chosen prior. The 95% credible upper limits on each amplitude are $\log \Omega_0^T < -9.8$, $\log \Omega_0^V < -9.7$, and $\log \Omega_0^S < -9.3$.

In Fig. 11, we show the posteriors obtained if
we additionally include design-sensitivity Advanced
Virgo, incorporating simulated measurements for the
Hanford-Virgo and Livingston-Virgo detector pairs. For
reference, the gray histograms show the posteriors from
Fig. 10 obtained by Advanced LIGO alone. The Advanced
LIGO-Virgo posteriors are virtually identical to those
obtained from Advanced LIGO alone, with 95% credible
upper limits of $\log \Omega_0^T < -9.9$, $\log \Omega_0^V < -9.6$, and
$\log \Omega_0^S < -9.4$. In the case of a null detection, then, the
inclusion of Advanced Virgo does not notably improve the
upper limits placed on the amplitudes of tensor, vector, and
scalar backgrounds.

*Case 2: Tensor background.*—Next, we produce a
simulated observation of a pure tensor background with
amplitude $\log \Omega_0^T = -8.78$ and spectral index $\alpha_T = 2/3$.
The amplitude is chosen such that the background would be
detected by Advanced LIGO with expected $\langle \text{SNR}_{\text{OPT}} \rangle = 5$
after 3 years of observation at design sensitivity. The
odds ratios obtained for this simulated observation are
$\ln \mathcal{O}_N^{\text{SIG}} = 8.4$ and $\ln \mathcal{O}_{\text{GR}}^{\text{NGR}} = -1.4$, indicating a strong
detection consistent with general relativity.

The corresponding parameter posteriors are shown in Fig. 12. In this case, the injected parameter values are shown via dot-dashed black lines. The $\log \Omega_0^T$ posterior is strongly peaked near the true value, with a central 68% credible interval of $-9.0 \le \log \Omega_0^T \le -8.7$ and a median value of $\log \Omega_0^T = -8.8$. The vector and scalar amplitudes, in turn, are consistent with the lower bound on our prior, with 95% credible upper limits of $\log \Omega_0^V < -9.2$ and $\log \Omega_0^S < -9.0$.

The parameter estimation results when additionally
including Advanced Virgo are given in Fig. 13. Once
again, the gray histograms show parameter estimation
results from Advanced LIGO alone. Although Virgo does
not notably improve our confidence in the detection, it can
serve to break degeneracies present between different
polarization modes. We begin to see this behavior in
Fig. 13, in which the vector and scalar log-amplitude
posteriors are pushed to smaller values in the joint
LIGO-Virgo analysis. When including Advanced Virgo,
we obtain a marginally tighter 68% credible interval of
$-8.9 \le \log \Omega_0^T \le -8.7$ on the tensor amplitude, and
slightly improved upper limits of $\log \Omega_0^V < -9.3$ and
$\log \Omega_0^S < -9.2$ on vector and scalar amplitudes.

*Case 3: Tensor and scalar backgrounds.*—As discussed above, most alternative theories of gravity would predict a stochastic background of mixed polarization. For our final case study, we therefore consider a mixed background with both tensor ($\log \Omega_0^T = -8.48$ and $\alpha_T = 2/3$) and scalar ($\log \Omega_0^S = -7.83$ and $\alpha_S = 0$) components. The amplitudes are chosen such that each component is individually observable with $\langle \text{SNR}_{\text{OPT}} \rangle = 10$ after 3 years of observation. Analysis with MultiNest yields odds ratios $\ln \mathcal{O}_N^{\text{SIG}} = 193.5$ and $\ln \mathcal{O}_{\text{GR}}^{\text{NGR}} = 16.1$, representing an extremely loud detection with very strong evidence for the presence alternative polarizations.

The posteriors obtained for this data are shown in Fig. 14. Despite the strength of the simulated stochastic signal, we see that parameter estimation results are dominated by degeneracies between the different polarization modes. Although the tensor and scalar amplitude posteriors are locally peaked about their true values, much of the background’s energy is misattributed to vector modes, illustrating that potentially severe degeneracies persist even at high SNRs. These degeneracies are exacerbated for backgrounds with small or negative spectral indices, as in the present case. Such backgrounds preferentially weight low frequencies where the Advanced LIGO overlap reduction functions are all similar (see Fig. 2). This example serves to illustrate that, while Advanced LIGO can likely identify the presence of alternative polarizations through the odds ratio $\mathcal{O}_{\text{GR}}^{\text{NGR}}$, Advanced LIGO alone is unable to determine which modes (vector or scalar) have been detected.

In contrast, the degeneracies in Fig. 14 are completely
broken with the inclusion of Advanced Virgo. Whereas the
$\Omega_0^V$ posterior is strongly peaked in Fig. 14, we see in Fig. 15
that the posterior is instead entirely consistent with our
<figure><img src="image_12.png" /><figcaption>FIG. 10. Posteriors obtained for a simulated Advanced LIGO observation of pure Gaussian noise (case 1 in Table I), under the TVS hypothesis. The subplots along the diagonal show marginalized posteriors for the amplitudes and slopes of the tensor, vector, and scalar backgrounds (blue, red, and green, respectively), while the remaining subplots show the 2D posterior between each pair of parameters. Above each marginalized posterior we quote the median posterior value and ±34% credible uncertainties. Each amplitude posterior is consistent with our lower prior bound, reflecting the nondetection of a stochastic background.</figcaption></figure>

<figure><img src="image_13.png" /><figcaption>FIG. 11. Marginalized amplitude and slope posteriors for the Gaussian noise observation in Fig. 10, after the additional inclusion of design-sensitivity Advanced Virgo. For reference, the light gray histograms show the Advanced-LIGO-only results from Fig. 10. As above, dashed gray lines show the priors placed on each parameter. We see that the inclusion of Advanced Virgo does not significantly affect the parameter estimation results.</figcaption></figure>
<figure><img src="image_14.png" /><figcaption>FIG. 12. As in Fig. 10, for a simulated observation of a pure tensor background (case 2 in Table I). The injected tensor amplitude and slope are indicated by dot-dashed black lines. The tensor amplitude and slope posteriors are peaked about their true values. The vector and scalar amplitude posteriors, meanwhile, are consistent with our lower prior bound.</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>FIG. 13. Marginalized amplitude and slope posteriors for the tensor background observation in Fig. 12, after the additional inclusion of design-sensitivity Advanced Virgo. For reference, the light gray histograms show the Advanced-LIGO-only results from Fig. 12. The joint LIGO-Virgo parameter estimation yields a slightly tighter measurement of $\Omega_0^T$, as well as somewhat improved upper limits on $\Omega_0^V$ and $\Omega_0^S$.</figcaption></figure>

lower prior bound when including Advanced Virgo. The tensor and scalar amplitude posteriors, meanwhile, are each more strongly peaked about their correct values and are now inconsistent with the lower amplitude bound. Thus,

while Advanced Virgo generally does not improve our ability to *detect* a stochastic background, we see that it can significantly improve prospects for simultaneous parameter estimation of multiple polarizations.
<figure><img src="image_16.png" /><figcaption>FIG. 14. As in Figs. 10 and 12, for a simulated observation of a mixed tensor and scalar background (case 3 in Table I). While the $\Omega_0^T$ and $\Omega_0^S$ posteriors are locally peaked about the true values, much of the observed energy is mistaken for vector polarizations. Thus, Advanced LIGO alone is unable to break the degeneracy between tensor, vector, and scalar amplitudes.</figcaption></figure>

<figure><img src="image_17.png" /><figcaption>FIG. 15. Marginalized amplitude and slope posteriors for the mixed tensor and scalar background observation in Fig. 14, after the additional inclusion of design-sensitivity Advanced Virgo. For reference, the light gray histograms show the Advanced-LIGO-only results from Fig. 14. In contrast to the results in Fig. 14, the degeneracy between polarization modes is completely broken when including Advanced Virgo. Thus, while Advanced Virgo does not particularly improve prospects for the detection of a mixed background, it can significantly improve our ability to perform parameter estimation on multiple modes simultaneously.</figcaption></figure>
## VI. BROKEN TENSOR SPECTRA

The stochastic search we present here offers a means to search for alternative gravitational-wave polarizations in a nearly model-independent way. Unlike direct searches for compact binary coalescences, our search makes minimal assumptions about the source and nature of the stochastic background. We do, however, make one notable assumption: that the energy density spectra $\Omega^a(f)$ are well described by power laws in the Advanced LIGO frequency band. This is expected to be a reasonable approximation for most predicted astrophysical sources of gravitational waves. The backgrounds expected from stellar-mass binary black holes [76], core-collapse supernovae [40], and rotating neutron stars [88–90], for instance, are all well modeled by power laws in the Advanced LIGO band. It may be, however, that the stochastic background is in fact not well described by a single power law. This may be the case if, for instance, the background is dominated by high-mass binary black holes, an excess of systems at high redshift, or previously unexpected sources of gravitational waves [76].

Given that our search allows only for power-law background models, how would we interpret a non-power-law background? In particular, if the stochastic background is purely tensorial (obeying general relativity) but is not well described by a power law, would our search mistakenly claim evidence for alternative polarizations?

To investigate this question, we consider simulated Advanced LIGO observations of pure tensor backgrounds described by broken power laws:

$$ \Omega^T(f) = \begin{cases} \Omega_0 \left(\frac{f}{f_k}\right)^{\alpha_1} & (f < f_k) \\ \Omega_0 \left(\frac{f}{f_k}\right)^{\alpha_2} & (f \ge f_k). \end{cases} \qquad (13) $$

Here, $\Omega_0$ is the background’s amplitude at the “knee frequency” $f_k$, while $\alpha_1$ and $\alpha_2$ are the slopes below and above the knee frequency, respectively. We set the knee frequency to $f_k = 30$ Hz, placing the backgrounds’ knees in the most sensitive band of the stochastic search. The odds ratios $\mathcal{O}_{\text{GR}}^{\text{NGR}}$ we obtain for these broken power laws are shown in Fig. 16 as a function of the two slopes $\alpha_1$ and $\alpha_2$. Each simulation assumes 3 years of observation at design sensitivity, and the amplitudes $\Omega_0$ are scaled such that each background has expected $\langle \text{SNR}_{\text{OPT}} \rangle = 5$ after this time. Any trends in Fig. 16 are therefore due to the backgrounds’ spectral shapes rather than their amplitudes.

If tensor broken power laws are indeed misclassified by our search, we should expect large, positive $\ln \mathcal{O}_{\text{GR}}^{\text{NGR}}$ values in Fig. 16. Instead, we see that broken power laws are not systematically misclassified. When $\alpha_1$ and $\alpha_2$ are each positive, we recover $\ln \mathcal{O}_{\text{GR}}^{\text{NGR}} \approx -1.5$, correctly classifying backgrounds as tensorial despite the fact that they are not described by power laws. When $\alpha_1 < 0$, meanwhile, we recover odds ratios scattered about $\ln \mathcal{O}_{\text{GR}}^{\text{NGR}} \approx 0$. This

<figure><img src="image_18.png" /><figcaption>FIG. 16. Odds ratios $\mathcal{O}_{\text{GR}}^{\text{NGR}}$ obtained for simulated Advanced LIGO observations of tensor-polarized broken power-law backgrounds with energy density spectra given by Eq. (13). The parameters $\alpha_1$ and $\alpha_2$ are the backgrounds’ slopes below and above the “knee” frequency $f_k$, which we take to be 30 Hz (in the center of the stochastic sensitivity band). We scale the amplitude $\Omega_0$ of each background such that it is optimally detectable with $\langle \text{SNR}_{\text{OPT}} \rangle = 5$ after the simulated observation period. By design, these backgrounds are not well described by single power laws, the form explicitly assumed in our search. Despite this fact, we find that these backgrounds are not systematically misclassified as containing vector or scalar polarization.</figcaption></figure>

simply reflects the fact that when $\alpha_1$ is negative, the majority of a background’s SNR is collected at low frequencies where Advanced LIGO’s tensor, vector, and scalar overlap reduction functions are degenerate. In such a case we do not show preference for either model over the other. Note that we find $\ln \mathcal{O}_{\text{GR}}^{\text{NGR}} \approx 0$ even along the line $\alpha_2 = \alpha_1$ (for $\alpha_1 < 0$), where the background is described by a single power law.

We expect broken power laws to be most problematic when $\alpha_1 > 0$ and $\alpha_2 < 0$; in this case, a background’s SNR is dominated by a small frequency band around the knee itself. This would be the case if, for instance, the stochastic background were dominated by unexpectedly massive binary black hole mergers [76]. Figure 16 does suggest a larger scatter in $\log \mathcal{O}_{\text{GR}}^{\text{NGR}}$ for such backgrounds. Even in this region, however, there is not a systematic bias towards larger values of $\mathcal{O}_{\text{GR}}^{\text{NGR}}$, and the largest recovered odds ratios have $\log \mathcal{O}_{\text{GR}}^{\text{NGR}} \lesssim 2.5$, well below the level required to confidently claim evidence for the presence of alternative polarizations.

Despite the fact that we assume purely power-law models for the stochastic energy-density spectra, our search appears reasonably robust against broken power-law spectra that are otherwise purely tensor polarized. In particular, in order to be mistakenly classified by our search, a tensor stochastic background would have to emulate the pattern of positive and negative cross power associated with the vector and/or scalar overlap reduction functions (see, for
instance, Fig. 5). This is simply not easy to do without a
pathological background. While we have demonstrated this
only for Advanced LIGO, we find similarly robust results
for three-detector Advanced LIGO-Virgo observations.

Nevertheless, when interpreting odds ratios $O_{\text{GR}}^{\text{NGR}}$ it should be kept in mind that the true stochastic background may deviate from a power law. Even if a broken tensor background is not misclassified in our analysis, the parameter estimation results we obtain would likely be incorrect (another example of so-called “stealth bias”). It should be pointed out, though, that our analysis is not fundamentally restricted to power-law models. While we adopt power-law models here for computational simplicity, our analysis can be straightforwardly expanded in the future to include more complex models for the stochastic energy-density spectrum.

VII. DISCUSSION

The direct detection of gravitational waves by Advanced LIGO and Virgo has opened up new and unique prospects for testing general relativity. One such avenue is the search for vector and scalar gravitational-wave polarizations, predicted by some alternative theories of gravity but prohibited by general relativity. Observation of vector or scalar polarizations in the stochastic background would therefore represent a clear violation of general relativity. While the first preliminary measurements have recently been made of the polarization of GW170814, our ability to study the polarization of transient gravitational-wave signals is currently limited by the number and orientation of current-generation detectors. In contrast, searches for long-duration sources like the stochastic background offer a promising means of directly measuring gravitational-wave polarizations with existing detectors.

In this paper, we explored a procedure by which
Advanced LIGO can detect or constrain the presence of
vector and scalar polarizations in the stochastic back-
ground. In Sec. III, we found that a stochastic background
dominated by alternative polarization modes may be
missed by current searches optimized only for tensor
polarizations. In particular, backgrounds of vector and
scalar polarizations with large, positive slopes may take
up to 10 times as long to detect with current methods,
relative to a search optimized for alternative polarizations.
In Sec. IV, we therefore proposed a Bayesian method with
which to detect a generically polarized stochastic back-
ground. This method relies on the construction of two odds
ratios (see Appendix C). The first serves to determine if a
stochastic background has been detected, while the second
quantifies evidence for the presence of alternative polar-
izations in the background. This search has the advantage
of being entirely generic; it is capable of detecting and
identifying stochastic backgrounds containing any combi-
nation of gravitational-wave polarizations. With this
method, we demonstrated that flat scalar-polarized

backgrounds of amplitude $\Omega_0^S \approx 2 \times 10^{-8}$ can be confi-
dently identified as nontensorial with Advanced LIGO.

In Sec. V, we then considered the ability of Advanced LIGO to perform simultaneous parameter estimation on tensor, vector, and scalar components of the stochastic background. After 3 years of observation at design sensitivity, Advanced LIGO will be able to limit the amplitudes of tensor, vector, and scalar polarizations to $\Omega_0^T < 1.6 \times 10^{-10}$, $\Omega_0^V < 2.0 \times 10^{-10}$, and $\Omega_0^S < 5.0 \times 10^{-10}$, respectively, at 95% credibility. If, however, a stochastic background of mixed polarization is detected, Advanced LIGO alone cannot precisely determine the parameters of the tensor, vector, and/or scalar components simultaneously due to large degeneracies between modes.

We also considered how the addition of Advanced Virgo to the Hanford-Livingston network affects the search for alternative polarizations. In Sec. IV, we found that addition of Advanced Virgo does not particularly increase our ability to detect or identify backgrounds of alternative polarizations. However, we demonstrated in Sec. V that Advanced Virgo *does* significantly improve our ability to perform parameter estimation on power-law backgrounds, breaking the degeneracies that plagued the Hanford-Livingston analysis.

Relative to other modeled searches for gravitational
waves, the stochastic search described here has the advan-
tage of being nearly model independent. We have, however,
made one large assumption: that the tensor, vector, and
scalar energy-density spectra are well described by power
laws in the Advanced LIGO band. Finally, in Sec. VI, we
explored the implications of this assumption, asking the
following question: Would tensor backgrounds *not*
described by power laws be mistaken for alternative
polarizations in our search? We found that our proposed
Bayesian method is reasonably robust against this pos-
sibility. In particular, even pure tensor backgrounds with
sharply broken power-law spectra are not systematically
misidentified by our search.

The nondetection of alternative polarizations in the stochastic background may yield interesting experimental constraints on extended theories of gravity. Meanwhile, any experimental evidence for alternative polarizations in the stochastic background would be a remarkable step forward for experimental tests of gravity. Of course, if future stochastic searches do yield evidence for alternative polarizations, careful study would be required to verify that this result is not due to unmodeled effects like non-Gaussianity or anisotropy in the stochastic background [27,91–94]. Comparison to polarization measurements of other long-lived sources like rotating neutron stars [33,34] will additionally aid in the interpretation of stochastic search results.

Several future developments may further improve the
ability of ground-based detectors to detect alternative
polarization modes in the stochastic background. First,
the continued expansion of the ground-based detector
network will improve our ability to both resolve the stochastic background and accurately determine its polarization content. Second, while we presently assume that the stochastic background is Gaussian, the background contribution from binary black holes is expected to be highly non-Gaussian [26]. Future stochastic searches may therefore be aided by the development of novel data analysis techniques optimized for non-Gaussian backgrounds [92–94].

## ACKNOWLEDGMENTS

We would like to thank Thomas Dent, Gregg Harry, Joe Romano, and Alan Weinstein for their careful reading of this manuscript, as well as many members of the LIGO-Virgo Collaboration Stochastic Backgrounds working group for helpful comments and conversation. T. C. and M. I. are members of the LIGO Laboratory, supported by funding from the U.S. National Science Foundation. LIGO was constructed by the California Institute of Technology and Massachusetts Institute of Technology with funding from the National Science Foundation and operates under Cooperative Agreement No. PHY-0757058. N. C. and J. T. are supported by NSF Grant No. PHY-1505373. The work of A. M. was supported in part by the NSF Grant No. PHY-1204944 at the University of Minnesota. M. S. is partially supported by STFC (UK) under the Research Grant No. ST/L000326/1. E. T. is supported through ARC FT150100281 and CE170100004. This paper carries the LIGO document number LIGO-P1700059 and King’s College London report number KCL-PH-TH/2017-25.

## APPENDIX A: OVERLAP REDUCTION FUNCTIONS

The sensitivity of a two-detector network to a stochastic gravitational-wave background is quantified by the overlap reduction function [70,71],

$$ \gamma(f) \propto \sum_A \int e^{2\pi i f \hat{\Omega} \cdot \Delta x / c} F_1^A(\hat{\Omega}) F_2^A(\hat{\Omega}) d\Omega, \quad (A1) $$

where $\Delta x$ is the displacement vector between detectors, c is the speed of light, and $F_{1/2}^A(\hat{\Omega})$ are the antenna patterns describing the response of each detector to gravitational waves of polarization A propagating from the direction $\hat{\Omega}$. The overlap reduction function is effectively the sky-averaged product of the two detectors' antenna patterns, weighted by the additional phase accumulated as a gravitational wave propagates from one site to the other.

In the standard stochastic search, the summation in Eq. (A1) is taken over the tensor plus and cross polarizations. When extending the stochastic search to generic gravitational-wave polarizations, we now must consider three separate overlap reduction functions for the tensor, vector, and scalar modes [30]:

$$ \gamma_T(f) = \frac{5}{8\pi} \sum_{A=\{+,\times\}} \int e^{2\pi i f \hat{\Omega} \cdot \Delta x / c} F_1^A(\hat{\Omega}) F_2^A(\hat{\Omega}) d\Omega, $$

$$ \gamma_V(f) = \frac{5}{8\pi} \sum_{A=\{x,y\}} \int e^{2\pi i f \hat{\Omega} \cdot \Delta x / c} F_1^A(\hat{\Omega}) F_2^A(\hat{\Omega}) d\Omega, $$

$$ \gamma_S(f) = \frac{5}{8\pi} \sum_{A=\{b,l\}} \int e^{2\pi i f \hat{\Omega} \cdot \Delta x / c} F_1^A(\hat{\Omega}) F_2^A(\hat{\Omega}) d\Omega. \quad (A2) $$

We normalize these functions such that $\gamma_T(f) = 1$ for coincident, coaligned detectors; detectors that are rotated or separated relative to one another have $\gamma_T(f) < 1$. The amplitudes of $\gamma_V(f)$ and $\gamma_S(f)$, meanwhile, express relative sensitivities to vector and scalar backgrounds.

Note that the normalization of $\gamma_S(f)$ differs from that of Nishizawa et al. in Ref. [30]. This difference is due to the definition of Nishizawa et al. of the longitudinal polarization tensor as

$$ \hat{\mathbf{e}}^{l} = \sqrt{2} \hat{\Omega} \otimes \hat{\Omega}, \quad (A3) $$

rather than the more common [12]

$$ \hat{\mathbf{e}}^{l} = \hat{\Omega} \otimes \hat{\Omega} \quad (A4) $$

(to distinguish between these two conventions, the quantities adopted by Nishizawa et al. are underscored with tildes). As a consequence, Nishizawa et al. obtain a longitudinal antenna pattern,

$$ F^{l}(\hat{\Omega}) = \frac{1}{\sqrt{2}} \sin^2 \theta \cos 2\phi, \quad (A5) $$

which differs by a factor of $\sqrt{2}$ from the conventional form [12]:

$$ F^l(\hat{\Omega}) = \frac{1}{2} \sin^2 \theta \cos 2\phi. \quad (A6) $$

Correspondingly, the quantity $\hat{\Omega}^l(f)$ defined by Nishizawa et al. is actually half of the canonical energy density in longitudinal gravitational waves, as defined by Eqs. (2) and (4):

$$ \hat{\Omega}^l(f) = \frac{1}{2} \hat{\Omega}^l(f). \quad (A7) $$

While each overlap reduction functions may be calculated numerically via Eq. (A2), they may also be analytically expanded in terms of spherical Bessel functions [30,70]. See Ref. [30] for definitions of the tensor, vector, and scalar overlap reduction functions in this analytic form. Note, however, that these definitions follow the normalization convention of Nishizawa et al., as discussed above;
the analytic expression given for $\gamma_S(f)$ must be divided by 3 to match our Eq. (A2).

## APPENDIX B: OPTIMAL SIGNAL-TO-NOISE RATIO

Searches for the stochastic background rely on measurements $\hat{C}(f)$ of the cross power between two detectors. As discussed in Sec. III, the expectation value and variance of $\hat{C}(f)$ are given by Eqs. (5) and (6), respectively. Here, we derive the optimal broadband signal-to-noise ratio [Eq. (9)], which combines a spectrum of cross-correlation measurements into a single detection statistic.

Given a measured spectrum $\hat{C}(f)$ and associated uncertainties $\sigma^2(f)$, a single broadband statistic may be formed via the weighted sum

$$ \hat{C} = \frac{\sum_f \hat{C}(f) w(f) / \sigma^2(f)}{\sum_f w(f) / \sigma^2(f)}, \quad (B1) $$

where $w(f)$ is a set of yet-undefined weights. The mean and variance of $\hat{C}$ are

$$ \langle \hat{C} \rangle = \frac{\sum_f \gamma_a(f) \Omega^a(f) w(f) / \sigma^2(f)}{\sum_f w(f) / \sigma^2(f)} \quad (B2) $$

and

$$ \sigma^2 = \frac{\sum_f w^2(f) / \sigma^2(f)}{\left[ \sum_f w(f) / \sigma^2(f) \right]^2}, \quad (B3) $$

where $\gamma_a(f)\Omega^a(f)$ denotes summation $\sum_a \gamma_a(f)\Omega^a(f)$ over polarization modes $a \in \{T, V, S\}$.

We define a broadband signal-to-noise ratio by $\text{SNR} = \hat{C}/\sigma$. In the limit $df \to 0$, this quantity may be written

$$ \text{SNR} = \frac{(\hat{C}|w)}{\sqrt{(w|w)}}, \quad (B4) $$

where we substitute Eq. (6) for $\sigma^2(f)$ and make use of the inner product defined in Eq. (8). The expected SNR is maximized when the chosen weights are equal to the true background, such that $w(f) = \gamma_a(f)\Omega^a(f)$. In this case, the optimal expected SNR of the stochastic background becomes

$$ \langle \text{SNR}_{\text{opt}} \rangle = \sqrt{(\gamma_a \Omega^a | \gamma_b \Omega^b)}. \quad (B5) $$

## APPENDIX C: ODDS RATIO CONSTRUCTION

Here, we describe the construction of the odds ratios $O_N^{\text{SIG}}$ and $O_{\text{GR}}^{\text{NGR}}$ introduced in Sec. IV. Given data $\hat{C}(f)$, the

Bayesian evidence for some hypothesis $\mathcal{A}$ with parameters $\theta_A$ is defined as

$$ P(\hat{C}|\mathcal{A}) = \int \mathcal{L}(\hat{C}|\theta_A, \mathcal{A})\pi(\theta_A|\mathcal{A})d\theta_A. \quad (C1) $$

Here, the likelihood $\mathcal{L}(\hat{C}|\theta_A, \mathcal{A})$ gives the conditional probability of the measured data under hypothesis $\mathcal{A}$ for fixed parameter values, while $\pi(\theta_A|\mathcal{A})$ is the prior probability set on these parameters. When selecting between two such hypotheses $\mathcal{A}$ and $\mathcal{B}$, we may define an odds ratio

$$ \mathcal{O}_{\mathcal{B}}^{\mathcal{A}} = \frac{P(\hat{C}|\mathcal{A}) \pi(\mathcal{A})}{P(\hat{C}|\mathcal{B}) \pi(\mathcal{B})}. \quad (C2) $$

The first factor in Eq. (C2), called the Bayes factor, is the ratio between the Bayesian evidences for hypotheses $\mathcal{A}$ and $\mathcal{B}$. The second term, meanwhile, is the ratio between the prior probabilities $\pi(\mathcal{A})$ and $\pi(\mathcal{B})$ assigned to each hypothesis.

To construct odds ratios for our stochastic background analysis, we first need the likelihood $\mathcal{L}(\{\hat{C}\}|\theta, \mathcal{A})$ of a measured cross-power spectrum under model $\mathcal{A}$ with some parameters $\theta$. In the presence of Gaussian noise, the likelihood of measuring a specific $\hat{C}(f)$ within a single frequency bin is [70,76,95]

$$ \mathcal{L}[\hat{C}(f)|\theta, \mathcal{A}] \propto \exp \left( - \frac{[\hat{C}(f) - \gamma_a(f)\Omega_A^a(\theta; f)]^2}{2\sigma^2(f)} \right), \quad (C3) $$

with variance $\sigma^2(f)$ given by Eq. (6). Here, $\Omega_A^a(\theta; f)$ is our model for the energy-density spectrum under hypothesis $\mathcal{A}$ and with parameters $\theta$, evaluated at the given frequency $f$. The full likelihood $\mathcal{L}(\{\hat{C}\}|\theta, \mathcal{A})$ for a spectrum of cross-correlation measurements is the product of the individual likelihoods in each frequency bin:

$$ \begin{aligned} \mathcal{L}(\{\hat{C}\}|\theta, \mathcal{A}) &\propto \prod_f \mathcal{L}[\hat{C}(f)|\theta, \mathcal{A}] \\ &= \mathcal{N} \exp \left[ -\frac{1}{2} (\hat{C} - \gamma_a \Omega_A^a | \hat{C} - \gamma_b \Omega_A^b) \right], \end{aligned} \quad (C4) $$

where $\mathcal{N}$ is a normalization coefficient and we use the inner product defined by Eq. (8).

As discussed in Sec. IV, we seek to detect and characterize a generic stochastic background via the construction of two odds ratios: $O_N^{\text{SIG}}$, which indicates whether a background of any polarization is present, and $O_{\text{GR}}^{\text{NGR}}$, which quantifies evidence for the presence of alternative polarization modes. First consider $O_N^{\text{SIG}}$. Under the noise hypothesis (N), we assume that no signal is present [such that
$\Omega_N^a(f) = 0$]. From Eq. (C4), the corresponding likelihood is simply

$$ \mathcal{L}(\{\hat{C}\}|N) = \mathcal{N} \exp \left[ -\frac{1}{2} (\hat{C}|\hat{C}) \right]. \quad (C5) $$

The signal hypothesis (SIG) is somewhat more complex. The signal hypothesis is ultimately the union of seven distinct subhypotheses that together describe all possible combinations of tensor, vector, and scalar polarizations [34,96]. To understand this, first define a “TVS” hypothesis that allows for the simultaneous presence of tensor, vector, and scalar polarization. In this case, we model the stochastic energy-density spectrum as a sum of three power laws,

$$ \Omega_{\text{TVS}}(f) = \Omega_0^T \left(\frac{f}{f_0}\right)^{\alpha_T} + \Omega_0^V \left(\frac{f}{f_0}\right)^{\alpha_V} + \Omega_0^S \left(\frac{f}{f_0}\right)^{\alpha_S}, \quad (C6) $$

with free parameters $\Omega_0^a$ and $\alpha_a$ setting the amplitude and spectral index of each polarization sector. The priors on these parameters are given by Eqs. (C11) and (C12).

In defining the TVS hypothesis, we make the explicit assumption that tensor, vector, and scalar radiation are each present. This is not the only possibility, of course. A second distinct hypothesis, for instance, is that only tensor and vector polarizations exist. This is our “TV” hypothesis. We model the corresponding energy spectrum as

$$ \Omega_{\text{TV}}(f) = \Omega_0^T \left(\frac{f}{f_0}\right)^{\alpha_T} + \Omega_0^V \left(\frac{f}{f_0}\right)^{\alpha_V}. \quad (C7) $$

In a similar fashion, we must ultimately define seven such hypotheses, denoted TVS, TV, TS, VS, T, V, and S, to encompass all combinations of tensor, vector, and scalar gravitational-wave backgrounds. Our complete signal hypothesis is given by the union of these seven subhypotheses [34,96]. For each signal subhypothesis, we adopt the log-amplitude and slope priors given in Eqs. (C11) and (C12).

Each of the signal subhypotheses are logically independent [34,96], and so the odds ratio $\mathcal{O}_N^{\text{SIG}}$ between signal and noise hypotheses is given by the sum of odds ratios between the noise hypothesis and each of the seven signal subhypotheses:

$$ \mathcal{O}_N^{\text{SIG}} = \sum_{A \in \{\text{T,V,S}, \dots\}} \mathcal{O}_N^A. \quad (C8) $$

As illustrated in Fig. 17, we assign equal prior probability to the signal and noise hypotheses. Within the signal hypothesis, we weight each of the signal subhypotheses equally, such that the prior odds between, e.g., the T and N hypothesis is $\pi(\text{T})/\pi(\text{N}) = 1/7$. We note that our choice of prior probabilities is not unique; there may exist other valid

<figure><img src="image_19.png" /><figcaption>FIG. 17. Illustration of the prior odds assigned to models and subhypotheses in the hierarchical Bayesian search for generically polarized stochastic backgrounds. When constructing $\mathcal{O}_N^{\text{SIG}}$, we assign equal prior probability to the noise and signal models, as well as equal probability to the seven signal subhypotheses $\{\text{T}, \dots, \text{TVS}\}$. Similarly, when constructing $\mathcal{O}_{\text{GR}}^{\text{NGR}}$, we give equal probability to the non-GR and GR models and identically weight the six non-GR subhypotheses $\{\text{V}, \dots, \text{TVS}\}$.</figcaption></figure>

choices as well. Our analysis can easily accommodate different choices of prior weight.

The odds ratio $\mathcal{O}_{\text{GR}}^{\text{NGR}}$ is constructed similarly. In this case, we are selecting between the hypothesis that the stochastic background is purely tensor polarized (GR) or the hypothesis that additional polarization modes are present (NGR). The GR hypothesis is identical to our tensor-only hypothesis T from above:

$$ \Omega_{\text{GR}}(f) = \Omega_0^T \left(\frac{f}{f_0}\right)^{\alpha_T}. \quad (C9) $$

The NGR hypothesis, on the other hand, is the union of the six signal subhypotheses that are inconsistent with general relativity: V, S, TV, TS, VS, and TVS. The complete odds ratio between the NGR and GR hypothesis is then

$$ \mathcal{O}_{\text{GR}}^{\text{NGR}} = \sum_{A \in \{\text{V,S,TV}, \dots\}} \mathcal{O}_T^A. \quad (C10) $$

As shown in Fig. 17, we assign equal priors to the GR and NGR hypotheses as well as identical priors to the six NGR subhypotheses.

In computing the odds ratios $\mathcal{O}_N^{\text{SIG}}$ and $\mathcal{O}_{\text{GR}}^{\text{NGR}}$, we also need priors for the various parameters governing each model for the stochastic background. In the various energy-density models presented above, we define two classes of parameters: amplitudes $\Omega_0^a$ and spectral indices $\alpha_a$ of the background’s various polarization components. For each amplitude parameter, we use the prior

$$ \pi(\Omega_0) \propto \begin{cases} 1/\Omega_0 & (\Omega_{\text{min}} \le \Omega_0 \le \Omega_{\text{max}}) \\ 0 & (\text{otherwise}). \end{cases} \quad (C11) $$

This corresponds to a uniform prior in the log amplitudes between $\log \Omega_{\text{min}}$ and $\log \Omega_{\text{max}}$. In order for this prior to be
normalizable, we cannot let it extend all the way to $\Omega_{min} = 0$ ($\log \Omega_{min} \to -\infty$). Instead, we must choose a finite lower bound. While this lower bound is somewhat arbitrary, our results depend only weakly on the specific choice of bound [34]. In this paper, we take $\Omega_{min} = 10^{-13}$, an amplitude that is indistinguishable from noise with Advanced LIGO. Our upper bound, meanwhile, is $\Omega_{max} = 10^{-6}$, consistent with upper limits placed by Initial LIGO and Virgo [77].

We adopt a triangular prior on $\alpha$, centered at zero:

$$ \pi(\alpha) = \begin{cases} \frac{1}{\alpha_{\max}} \left(1 - \frac{|\alpha|}{\alpha_{\max}}\right) & (|\alpha| \le \alpha_{\max}) \\ 0 & (\text{otherwise}). \end{cases} \quad (\text{C12}) $$

This prior has several desirable properties. First, it captures a natural tendency for spectral index posteriors to peak symmetrically about $\alpha = 0$. As a result, our $\alpha$ posteriors reliably recover this prior in the absence of informative data (see Fig. 10, for example). Second, this prior preferentially weights shallower energy-density spectra. This quantifies our expectation that the stochastic background's energy density be distributed somewhat uniformly across logarithmic frequency intervals (at least in the LIGO band), rather than entirely at very high or very low frequencies.

Alternatively, Eq. (C12) can be viewed as corresponding to equal priors on the stochastic background's strength at two different frequencies. To understand this, first note that $\alpha$ may be written as a function of background amplitudes $\Omega_0$ and $\Omega_1$ at two frequencies $f_0$ and $f_1$:

$$ \alpha(\Omega_0, \Omega_1) = \frac{\log(\Omega_1/\Omega_0)}{\log(f_1/f_0)}. \quad (\text{C13}) $$

The prior probability of a particular slope $\alpha$ is equal to the probability of drawing any two amplitudes $\Omega_0$ and $\Omega_1$ satisfying $\log(\Omega_1/\Omega_0) = \alpha \log(f_1/f_2)$. This is given by the integral

$$ \pi(\alpha) = \int \pi(\log\Omega_1)\pi[\log\Omega_0 = \log\Omega_1 - \alpha\log(f_1/f_0)]d\log\Omega_1. \quad (\text{C14}) $$

For simplicity, we set $f_1 = 10f_0$ [such that $\log(f_1/f_0) = 1$] and place identical log-uniform priors [Eq. (C11)] on each amplitude. Under these assumptions, Eq. (C14) yields Eq. (C12).

In Secs. IV and V, we additionally consider the performance of the three-detector Advanced LIGO-Virgo network. The Bayesian framework considered here is easily extended to accommodate multiple detector pairs. The three LIGO and Virgo detectors allow for the measurement of three cross-correlation spectra: $\hat{C}^{\text{HL}}(f)$, $\hat{C}^{\text{HV}}(f)$, and $\hat{C}^{\text{LV}}(f)$. In the small signal limit [$\Omega^a(f) \ll 1$], the correlations between these measurements vanish at leading order, and so the three baselines can be treated as

statistically independent [70]. We can therefore factorize the joint likelihood for the three sets:

$$ \begin{aligned} & \mathcal{L}(\{\hat{C}^{\text{HL}}, \hat{C}^{\text{HV}}, \hat{C}^{\text{LV}}\}|\theta, \mathcal{A}) \\ &= \mathcal{L}(\{\hat{C}^{\text{HL}}\}|\theta, \mathcal{A})\mathcal{L}(\{\hat{C}^{\text{HV}}\}|\theta, \mathcal{A})\mathcal{L}(\{\hat{C}^{\text{LV}}\}|\theta, \mathcal{A}) \\ &= \mathcal{N} \exp\left\{-\frac{1}{2}\left[(\hat{C}^{\text{HL}} - \gamma_a^{\text{HL}}\Omega_A^a)|\hat{C}^{\text{HL}} - \gamma_b^{\text{HL}}\Omega_A^b\right.\right. \\ & \qquad \left. + (\hat{C}^{\text{HV}} - \gamma_a^{\text{HV}}\Omega_A^a)|\hat{C}^{\text{HV}} - \gamma_b^{\text{HV}}\Omega_A^b)\right. \\ & \qquad \left. + (\hat{C}^{\text{LV}} - \gamma_a^{\text{LV}}\Omega_A^a)|\hat{C}^{\text{LV}} - \gamma_b^{\text{LV}}\Omega_A^b]\right\}, \end{aligned} \quad (\text{C15}) $$

substituting likelihoods of the form Eq. (C4) for each pair of detectors. Note that we explicitly distinguish between the overlap reduction functions for each baseline, and $\mathcal{N}$ is again a normalization constant. Other than the above change to the likelihood, all other details of the odds ratio construction are unchanged when including three detectors.

## APPENDIX D: VALUATING BAYESIAN EVIDENCES WITH MultiNest

Here, we summarize details associated with using MultiNest to evaluate Bayesian evidences for various models of the stochastic background. The MultiNest algorithm allows for several user-defined parameters, including the number $n$ of live points used to sample the prior volume and the sampling efficiency $\epsilon$, which governs the acceptance rate of new proposed live points (see, e.g., Ref. [83] for details). MultiNest also provides the option to run in default or importance nested sampling (INS) modes, each of which uses different methods to evaluate evidence [84].

To set the number of live points, we investigate the convergence of MultiNest's evidence estimates with increasing values of $n$. For a single simulated observation of a tensorial background (with amplitude $\Omega_0^T = 2 \times 10^{-8}$ and slope $\alpha_T = 2/3$), for instance, Fig. 18 shows the recovered evidence for the T hypothesis (see Appendix C) as a function of $n$, using both the default (blue) and INS modes (green). The results are reasonably stable for $n \gtrsim 1000$; we choose $n = 2000$ live points. Meanwhile, our recovered evidence estimates do not exhibit noticeable dependence on the sampling efficiency; we choose the recommended values $\epsilon = 0.3$ for evidence evaluation and $\epsilon = 0.8$ for parameter estimation [83].

In addition to computing Bayesian evidences, MultiNest also returns an estimate of the numerical error associated with each evidence calculation. See, for instance, the error bars in Fig. 18. To gauge the accuracy of these error estimates, we construct a single simulated Advanced LIGO observation of a purely tensorial stochastic background (again with $\Omega_0^T = 2 \times 10^{-8}$ and $\alpha_T = 2/3$). We then use MultiNest to compute the corresponding TVS evidence 500 times, in both default and INS modes.
<figure><img src="image_20.png" /><figcaption>FIG. 18. MultiNest Bayesian log-evidence for a single simulated stochastic background observation as a function of the number of live points chosen. For the simulated data, we assume a tensor-polarized background (with $\Omega_0^T = 2 \times 10^{-8}$ and $\alpha_T = 2/3$) observed for 1 year with design-sensitivity Advanced LIGO and compute evidence using the T model (see Sec. IV). Results are shown for both MultiNest's default and INS modes; also shown are the error estimates provided by each mode. To compute the results presented in this paper, we use $n = 2000$ live points.</figcaption></figure>

The resulting distributions of log-evidence are shown in Fig. 19. The dashed error bars show the averaged $\pm 1\sigma$ intervals reported by MultiNest, while the solid bars show the $\pm 1\sigma$ intervals obtained manually from the distributions. We see that the errors reported by MultiNest's default mode appear to accurately reflect

<figure><img src="image_21.png" /><figcaption>FIG. 19. Histograms of MultiNest log-evidence (for the TVS model; see Sec. IV) obtained by evaluating a single simulated data set 500 times in both the default and INS modes. To generate the simulated data, we assume a 1-year observation of a tensor background ($\Omega_0^T = 2 \times 10^{-8}$ and $\alpha_T = 2/3$) with design-sensitivity Advanced LIGO. The dashed error bars show the mean 68% confidence interval reported by each method, while the solid error bars show the true 68% confidence interval computed from the log-evidence distributions.</figcaption></figure>

the numerical error in the evidence calculation, while the errors reported by the INS mode are underestimated by a factor of $\sim 2$.

Additionally, Fig. 19 illustrates several systematic differences between the default and INS results. First, default mode appears significantly more precise than INS mode, giving rise to a much narrower distribution. Not only is the INS log-evidence distribution wider, but it exhibits a large tail extending several units in log-evidence above the mean. We find that similarly long tails also appear for other pairs of injected signals and recovered models. For this reason, we choose to use the default mode of MultiNest in all evidence calculations. Typical numerical errors in default mode are of order $\delta(\log \text{-evidence}) \sim 0.1$, and so the uncertainty associated with a log-odds ratio is $\delta(\ln \mathcal{O}) \sim \sqrt{2\delta(\log \text{-evidence})}$, again of order 0.1. Additionally, we see that the peaks of the default and INS distributions do not coincide. In general, the peaks of log-evidence distributions from the default and INS modes lie $\sim 0.3$ units apart. Thus, there may be additional systematic uncertainties in a given evidence calculation. However, as long as we consistently use one mode or the other (in our case, default mode), any uniform systematic offset in the log-evidence will simply cancel when we ultimately compute a log-odds ratio.

[1] J. Aasi et al. (LIGO Scientific Collaboration), Advanced LIGO, Classical Quantum Gravity 32, 074001 (2015).

[2] F. Acernese, M. Agathos, K. Agatsuma, D. Aisa, N. Allemandou, A. Allocca, J. Amarni, P. Astone, G. Balestri et al., Advanced Virgo: A Second-Generation Interferometric Gravitational Wave Detector, Classical Quantum Gravity 32, 024001 (2015).

[3] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), Observation of Gravitational Waves from a Binary Black Hole Merger, Phys. Rev. Lett. 116, 061102 (2016).

[4] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), GW151226: Observation of Gravitational Waves from a 22-Solar-Mass Binary Black Hole Coalescence, Phys. Rev. Lett. 116, 241103 (2016).

[5] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), Binary Black Hole Mergers in the First Advanced LIGO Observing Run, Phys. Rev. X 6, 041015 (2016).

[6] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), GW170104: Observation of a 50-Solar-Mass Binary Black Hole Coalescence at Redshift 0.2, Phys. Rev. Lett. 118, 221101 (2017).

[7] R. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), GW170814: A Three-Detector Observation of Gravitational Waves from a Binary Black Hole Coalescence, Phys. Rev. Lett. 119, 141101 (2017).

[8] B. P. Abbott, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. X. Adhikari, and V. B. Adya et al. (LIGO Scientific Collaboration and
Virgo Collaboration), GW170817: Observation of Gravitational Waves from a Binary Neutron Star Inspiral, Phys. Rev. Lett. 119, 161101 (2017).

[9] D. M. Eardley, D. L. Lee, A. P. Lightman, R. V. Wagoner, and C. M. Will, *Gravitational-Wave Observations as a Tool for Testing Relativistic Gravity*, Phys. Rev. Lett. 30, 884 (1973).

[10] D. M. Eardley, D. L. Lee, and A. P. Lightman, *Gravitational-Wave Observations as a Tool for Testing Relativistic Gravity*, Phys. Rev. D 8, 3308 (1973).

[11] J. Gair, M. Vallisneri, S. L. Larson, and J. G. Baker, *Testing General Relativity with Low-Frequency, Space-Based Gravitational-Wave Detectors*, Living Rev. Relativity 16, 7 (2013).

[12] C. M. Will, *The Confrontation between General Relativity and Experiment*, Living Rev. Relativity 17, 4 (2014).

[13] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), *Tests of General Relativity with GW150914*, Phys. Rev. Lett. 116, 221101 (2016).

[14] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), *Directly Comparing GW150914 with Numerical Solutions of Einstein's Equations for Binary Black Hole Coalescence*, Phys. Rev. D 94, 064035 (2016).

[15] C. de Rham, J. Tate Deskins, A. J. Tolley, and S.-Y. Zhou, *Graviton Mass Bounds*, Rev. Mod. Phys. 89, 025004 (2017).

[16] M. Isi and A. J. Weinstein, *Probing Gravitational Wave Polarizations with Signals from Compact Binary Coalescences*, arXiv:1710.03794.

[17] J. D. Romano and N. J. Cornish, *Detection Methods for Stochastic Gravitational-Wave Backgrounds: A Unified Treatment*, Living Rev. Relativity 20, 2 (2017).

[18] K. Chatzioannou, N. Yunes, and N. Cornish, *Model-Independent Test of General Relativity: An Extended Post-Einsteinian Framework with Complete Polarization Content*, Phys. Rev. D 86, 022004 (2012).

[19] Y. Aso, Y. Michimura, K. Somiya, M. Ando, O. Miyakawa, T. Sekiguchi, D. Tatsumi, and H. Yamamoto, *Interferometer Design of the KAGRA Gravitational Wave Detector*, Phys. Rev. D 88, 043007 (2013).

[20] B. Iyer, T. Souradeep, C. S. Unnikrishnan, S. Dhurandhar, S. Raja, A. Kumar, and A. Sengupta, LIGO-India Technical Report No. LIGO-M1100296, 2011, https://dcc.ligo.org/M1100296/public.

[21] P. A. Rosado, *Gravitational Wave Background from Binary Systems*, Phys. Rev. D 84, 084004 (2011).

[22] X.-J. Zhu, E. Howell, T. Regimbau, D. Blair, and Z.-H. Zhu, *Stochastic Gravitational Wave Background From Coalescing Binary Black Holes*, Astrophys. J. 739, 86 (2011).

[23] C. Wu, V. Mandic, and T. Regimbau, *Accessibility of the Gravitational-Wave Background Due to Binary Coalescences to Second and Third Generation Gravitational-Wave Detectors*, Phys. Rev. D 85, 104024 (2012).

[24] X.-J. Zhu, E. J. Howell, D. G. Blair, and Z.-H. Zhu, *On the Gravitational Wave Background from Compact Binary Coalescences in the Band of Ground-Based Interferometers*, Mon. Not. R. Astron. Soc. 431, 882 (2013).

[25] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), *Upper Limits on the Stochastic Gravitational-Wave Background from Advanced LIGO's First Observing Run*, Phys. Rev. Lett. 118, 121101 (2017).

[26] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), GW150914: Implications for the Stochastic Gravitational-Wave Background from Binary Black Holes, Phys. Rev. Lett. 116, 131102 (2016).

[27] B. P. Abbott, R. Abbott, T. D. Abbott, M. R. Abernathy, F. Acernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. X. Adhikari et al., *Directional Limits on Persistent Gravitational Waves from Advanced LIGO's First Observing Run*, Phys. Rev. Lett. 118, 121102 (2017).

[28] B. P. Abbott, R. Abbott, T. D. Abbott, F. Acernese, K. Ackley, C. Adams, T. Adams, P. Addesso, R. X. Adhikari, and V. B. Adya et al. (LIGO Scientific Collaboration and Virgo Collaboration), GW170817: Implications for the Stochastic Gravitational-Wave Background from Compact Binary Coalescences, arXiv:1710.05837.

[29] E. Berti, E. Barausse, V. Cardoso, L. Gualtieri, P. Pani, U. Sperhake, L. C. Stein, N. Wex, K. Yagi, T. Baker et al., *Testing General Relativity with Present and Future Astrophysical Observations*, Classical Quantum Gravity 32, 243001 (2015).

[30] A. Nishizawa, A. Taruya, K. Hayama, S. Kawamura, and M. A. Sakagami, *Probing Nontensorial Polarizations of Stochastic Gravitational-Wave Backgrounds with Ground-Based Laser Interferometers*, Phys. Rev. D 79, 082002 (2009).

[31] A. Nishizawa, A. Taruya, and S. Kawamura, *Cosmological Test of Gravity with Polarizations of Stochastic Gravitational Waves Around 0.1–1 Hz*, Phys. Rev. D 81, 104043 (2010).

[32] A. Nishizawa and K. Hayama, *Probing for Massive Stochastic Gravitational-Wave Background with a Detector Network*, Phys. Rev. D 88, 064005 (2013).

[33] M. Isi, A. J. Weinstein, C. Mead, and M. Pitkin, *Detecting Beyond-Einstein Polarizations of Continuous Gravitational Waves*, Phys. Rev. D 91, 082002 (2015).

[34] M. Isi, M. Pitkin, and A. J. Weinstein, *Probing Dynamical Gravity with the Polarization of Continuous Gravitational Waves*, Phys. Rev. D 96, 042001 (2017).

[35] C. Brans and R. H. Dicke, *Mach's Principle and a Relativistic Theory of Gravitation*, Phys. Rev. 124, 925 (1961).

[36] D. Gerosa, U. Sperhake, and C. D. Ott, *Numerical Simulations of Stellar Collapse in Scalar-Tensor Theories of Gravity*, Classical Quantum Gravity 33, 135002 (2016).

[37] S. E. Gossan, P. Sutton, A. Stuver, M. Zanolin, K. Gill, and C. D. Ott, *Observing Gravitational Waves from Core-Collapse Supernovae in the Advanced Detector Era*, Phys. Rev. D 93, 042002 (2016).

[38] B. P. Abbott et al. (LIGO Scientific Collaboration and Virgo Collaboration), First Targeted Search for Gravitational-Wave Bursts from Core-Collapse Supernovae in Data of First-Generation Laser Interferometer Detectors, Phys. Rev. D 94, 102001 (2016).

[39] S. M. Adams, C. S. Kochanek, J. F. Beacom, M. R. Vagins, and K. Z. Stanek, *Observing the Next Galactic Supernova*, Astrophys. J. 778, 164 (2013).

[40] K. Crocker, V. Mandic, T. Regimbau, K. Belczynski, W. Gladysz, K. Olive, T. Prestegard, and E. Vangioni, *Model of Gravitational-Wave Background from Advanced LIGO's First Observing Run*, Phys. Rev. Lett. 118, 121101 (2017).
the Stochastic Gravitational-Wave Background Due to Core Collapse to Black Holes, Phys. Rev. D **92**, 063005 (2015).

[41] K. Crocker, T. Prestegard, V. Mandic, T. Regimbau, K. Olive, and E. Vangioni, Systematic Study of the Stochastic Gravitational-Wave Background Due to Stellar Core Collapse, Phys. Rev. D **95**, 063015 (2017).

[42] T. Damour and G. Esposito-Faréese, Tensor-Multi-Scalar Theories of Gravitation, Classical Quantum Gravity **9**, 2093 (1992).

[43] T. Damour and G. Esposito-Farèse, Nonperturbative Strong Field Effects in Tensor-Scalar Theories of Gravitation, Phys. Rev. Lett. **70**, 2220 (1993).

[44] P. C. C. Freire, N. Wex, G. Esposito-Farèse, J. P. W. Verbiest, M. Bailes, B. A. Jacoby, M. Kramer, I. H. Stairs, J. Antoniadis, and G. H. Janssen, The Relativistic Pulsar-White Dwarf Binary PSR J1738+0333—II. The Most Stringent Test of Scalar-Tensor Gravity, Mon. Not. R. Astron. Soc. **423**, 3328 (2012).

[45] J. Antoniadis, P. C. C. Freire, N. Wex, T. M. Tauris, R. S. Lynch, M. H. van Kerkwijk, M. Kramer, C. Bassa, V. S. Dhillon, T. Driebe et al., A Massive Pulsar in a Compact Relativistic Binary, Science **340**, 1233232 (2013).

[46] K. Yagi, L. C. Stein, and N. Yunes, Challenging the Presence of Scalar Charge and Dipolar Radiation in Binary Pulsars, Phys. Rev. D **93**, 024010 (2016).

[47] E. Barausse, N. Yunes, and K. Chamberlain, Theory-Agnostic Constraints on Black-Hole Dipole Radiation with Multiband Gravitational-Wave Astrophysics, Phys. Rev. Lett. **116**, 241104 (2016).

[48] M. W. Horbatsch and C. P. Burgess, Cosmic Black-Hole Hair Growth and Quasar OJ287, J. Cosmol. Astropart. Phys. 05 (2012) 010.

[49] E. Berti, V. Cardoso, L. Gualtieri, M. Horbatsch, and U. Sperhake, Numerical Simulations of Single and Binary Black Holes in Scalar-Tensor Theories: Circumventing the No-Hair Theorem, Phys. Rev. D **87**, 124020 (2013).

[50] T. Damour and A. Vilenkin, Cosmic Strings and the String Dilaton, Phys. Rev. Lett. **78**, 2288 (1997).

[51] S. Ölmez, V. Mandic, and X. Siemens, Gravitational-Wave Stochastic Background from Kinks and Cusps on Cosmic Strings, Phys. Rev. D **81**, 104028 (2010).

[52] M. Maggiore, Gravitational Wave Experiments and Early Universe Cosmology, Phys. Rep. **331**, 283 (2000).

[53] C. Caprini, R. Durrer, and G. Servant, Gravitational Wave Generation from Bubble Collisions in First-Order Phase Transitions: An Analytic Approach, Phys. Rev. D **77**, 124015 (2008).

[54] C. Caprini, M. Hindmarsh, S. Huber, T. Konstandin, J. Kozaczuk, G. Nardini, J. M. No, A. Petiteau, P. Schwaller, G. Servant, and D. J. Weir, Science with the Space-Based Interferometer eLISA. II: Gravitational Waves from Cosmological Phase Transitions, J. Cosmol. Astropart. Phys. 04 (2016) 001.

[55] R. Tso, M. Isi, Y. Chen, and L. Stein, in Proceedings of the Seventh Meeting on CPT and Lorentz Symmetry, Bloomington, IN, 2016 (World Scientific, Singapore, 2016).

[56] A. De Felice, T. Nakamura, and T. Tanaka, Possible Existence of Viable Models of Bi-Gravity with Detectable Graviton Oscillations by Gravitational Wave Detectors, Prog. Theor. Exp. Phys. (2014) 043E01.

[57] T. Narikawa, K. Ueno, H. Tagoshi, T. Tanaka, N. Kanda, and T. Nakamura, Detectability of Bigravity with Graviton Oscillations Using Gravitational Wave Observations, Phys. Rev. D **91**, 062007 (2015).

[58] V. A. Kostelecky, Gravity, Lorentz Violation, and the Standard Model, Phys. Rev. D **69**, 105009 (2004).

[59] V. A. Kostelecky and M. Mewes, Testing Local Lorentz Invariance with Gravitational Waves, Phys. Lett. B **757**, 510 (2016).

[60] N. Yunes, K. Yagi, and F. Pretorius, Theoretical Physics Implications of the Binary Black-Hole Mergers GW150914 and GW151226, Phys. Rev. D **94**, 084002 (2016).

[61] T. Damour, Theoretical Aspects of the Equivalence Principle, Classical Quantum Gravity **29**, 184001 (2012).

[62] T. Damour and K. Nordtvedt, General Relativity as a Cosmological Attractor of Tensor-Scalar Theories, Phys. Rev. Lett. **70**, 2217 (1993).

[63] A. Joyce, B. Jain, J. Khoury, and M. Trodden, Beyond the Cosmological Standard Model, Phys. Rep. **568**, 1 (2015).

[64] R. Jackiw and S.-Y. Pi, Chern-Simons Modification of General Relativity, Phys. Rev. D **68**, 104012 (2003).

[65] S. Alexander and J. Martin, Birefringent Gravitational Waves and the Consistency Check of Inflation, Phys. Rev. D **71**, 063526 (2005).

[66] S. Alexander, L. S. Finn, and N. Yunes, Gravitational-Wave Probe of Effective Quantum Gravity, Phys. Rev. D **78**, 066005 (2008).

[67] S. Alexander and N. Yunes, Chern-Simons Modified General Relativity, Phys. Rep. **480**, 1 (2009).

[68] C. R. Contaldi, J. Magueijo, and L. Smolin, Anomalous Cosmic-Microwave-Background Polarization and Gravitational Chirality, Phys. Rev. Lett. **101**, 141101 (2008).

[69] M. Isi and L. Stein, Stochastic Gravitational-Wave Energy Density in Beyond-GR Gravity (to be published).

[70] B. Allen and J. D. Romano, Detecting a Stochastic Background of Gravitational Radiation: Signal Processing Strategies and Sensitivities, Phys. Rev. D **59**, 102001 (1999).

[71] N. Christensen, Measuring the Stochastic Gravitational-Radiation Background with Laser-Interferometric Antennas, Phys. Rev. D **46**, 5250 (1992).

[72] M. E. S. Alves and M. Tinto, Pulsar Timing Sensitivities to Gravitational Waves from Relativistic Metric Theories of Gravity, Phys. Rev. D **83**, 123529 (2011).

[73] S. J. Chamberlin and X. Siemens, Stochastic Backgrounds in Alternative Theories of Gravity: Overlap Reduction Functions for Pulsar Timing Arrays, Phys. Rev. D **85**, 082001 (2012).

[74] G. Cella, C. N. Colacino, E. Cuoco, A. Di Virgilio, T. Regimbau, E. L. Robinson, and J. T. Whelan, Prospects for Stochastic Background Searches Using Virgo and LSC Interferometers, Classical Quantum Gravity **24**, S639 (2007).

[75] R. A. Isaacson, Gravitational Radiation in the Limit of High Frequency. II. Nonlinear Terms and the Effective Stress Tensor, Phys. Rev. **166**, 1272 (1968).

[76] T. Callister, L. Sammut, S. Qiu, I. Mandel, and E. Thrane, Limits of Astrophysics with Gravitational-Wave Backgrounds, Phys. Rev. X **6**, 031018 (2016).
[77] J. Aasi et al. (LIGO Scientific Collaboration and Virgo Collaboration), Improved Upper Limits on the Stochastic Gravitational-Wave Background from 2009–2010 LIGO and Virgo Data, Phys. Rev. Lett. 113, 231101 (2014).

[78] E. Thrane and J. D. Romano, Sensitivity Curves for Searches for Gravitational-Wave Backgrounds, Phys. Rev. D 88, 124032 (2013).

[79] M. Vallisneri and N. Yunes, Stealth Bias in Gravitational-Wave Parameter Estimation, Phys. Rev. D 87, 102002 (2013).

[80] S. Vitale and W. Del Pozzo, How Serious Can the Stealth Bias Be in Gravitational Wave Parameter Estimation?, Phys. Rev. D 89, 022002 (2014).

[81] J. Buchner, A. Georgakakis, K. Nandra, L. Hsu, C. Rangel, M. Brightman, A. Merloni, M. Salvato, J. Donley, and D. Kocevski, X-Ray Spectral Modelling of the AGN Obscuring Region in the CDFS: Bayesian Model Selection and Catalogue, Astron. Astrophys. 564, A125 (2014).

[82] F. Feroz and M. P. Hobson, Multimodal Nested Sampling: An Efficient and Robust Alternative to Markov Chain Monte Carlo Methods for Astronomical Data Analyses, Mon. Not. R. Astron. Soc. 384, 449 (2008).

[83] F. Feroz, M. P. Hobson, and M. Bridges, MultiNest: An Efficient and Robust Bayesian Inference Tool for Cosmology and Particle Physics, Mon. Not. R. Astron. Soc. 398, 1601 (2009).

[84] F. Feroz, M. P. Hobson, E. Cameron, and A. N. Pettitt, Importance Nested Sampling and the MultiNest Algorithm, arXiv:1306.2144.

[85] J. Skilling, Nested Sampling, AIP Conf. Proc. 735, 395 (2004).

[86] J. Skilling, Nested Sampling for General Bayesian Computation, Bayesian Anal. 1, 833 (2006).

[87] M. Vallisneri, Use and Abuse of the Fisher Information Matrix in the Assessment of Gravitational-Wave Parameter-Estimation Prospects, Phys. Rev. D 77, 042001 (2008).

[88] T. Regimbau and V. Mandic, Astrophysical Sources of a Stochastic Gravitational-Wave Background, Classical Quantum Gravity 25, 184018 (2008).

[89] C.-J. Wu, V. Mandic, and T. Regimbau, Accessibility of the Stochastic Gravitational Wave Background from Magnetars to the Interferometric Gravitational Wave Detectors, Phys. Rev. D 87, 042002 (2013).

[90] D. Talukder, E. Thrane, S. Bose, and T. Regimbau, Measuring Neutron-Star Ellipticity with Measurements of the Stochastic Gravitational-Wave Background, Phys. Rev. D 89, 123008 (2014).

[91] D. Meacher, E. Thrane, and T. Regimbau, Statistical Properties of Astrophysical Gravitational-Wave Backgrounds, Phys. Rev. D 89, 084063 (2014).

[92] L. Martellini and T. Regimbau, Semiparametric Approach to the Detection of Non-Gaussian Gravitational Wave Stochastic Backgrounds, Phys. Rev. D 89, 124009 (2014).

[93] E. Thrane, Measuring the Non-Gaussian Stochastic Gravitational-Wave Background: A Method for Realistic Interferometer Data, Phys. Rev. D 87, 043009 (2013).

[94] N. J. Cornish and J. D. Romano, When is a Gravitational-Wave Signal Stochastic?, Phys. Rev. D 92, 042001 (2015).

[95] V. Mandic, E. Thrane, S. Giampanis, and T. Regimbau, Parameter Estimation in Searches for the Stochastic Gravitational-Wave Background, Phys. Rev. Lett. 109, 171102 (2012).

[96] T. G. F. Li, W. Del Pozzo, S. Vitale, C. Van Den Broeck, M. Agathos, J. Veitch, K. Grover, T. Sidery, R. Sturani, and A. Vecchio, Towards a Generic Test of the Strong Field Dynamics of General Relativity Using Compact Binary Coalescence, Phys. Rev. D 85, 082003 (2012).