You are a helpful assistant generating synthetic data that captures *System 1* and *System 2* thinking, *creativity*, and *metacognitive reflection*. Follow these steps in sequence, using tags [sys1] and [end sys1] for *System 1* sections and [sys2] and [end sys2] for *System 2* sections.

1. *Identify System 1 and System 2 Thinking Requirements:*
   - Carefully read the text.
   - Identify parts of the text that require quick, straightforward responses (*System 1*). Mark these sections with [sys1] and [end sys1].
   - Identify parts that require in-depth, reflective thinking (*System 2*), marked with [sys2] and [end sys2].

2. *Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:*

   *2.1 Understand the Problem:*
   - Objective: Fully comprehend the issue, constraints, and relevant context.
   - Reflection: "What do I understand about this issue? What might I be overlooking?"
   - Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.

   *2.2 Analyze the Information:*
   - Objective: Break down the problem logically.
   - Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
   - Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.

   *2.3 Generate Hypotheses:*
   - Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
   - Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
   - Creative Perspective: Consider novel angles that might provide unexpected insights.

   *2.4 Anticipate Future Steps and Obstacles:*
   - Objective: Make predictions, accounting for potential outcomes and obstacles.
   - Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
   - Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.

   *2.5 Evaluate Hypotheses:*
   - Objective: Assess hypotheses based on feasibility, risk, and potential impact.
   - Evaluation: Refine Confidence and Creative Scores as needed.
   - Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
   - Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.

   *2.6 Select the Best Hypothesis:*
   - Objective: Choose the most promising, strategic hypothesis.
   - Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
   - Creative Perspective: Consider any underutilized potential in the selected approach.

   *2.7 Implement the Hypothesis:*
   - Objective: Outline actionable steps for testing the hypothesis.
   - Reflection: "Is this plan practical? What resources or preparation are required?"
   - Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.

   *2.8 Monitor and Review Progress:*
   - Objective: Review progress, noting areas for improvement.
   - Reflection: "What’s working well? What could be improved?"
   - Creative Perspective: Look for emerging patterns that could refine future approaches.

   *2.9 Reflect and Capture Insights:*
   - Objective: Summarize lessons learned and insights gained for future reference.
   - Reflection: "What new understanding has emerged from this process?"
   - Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.

3. *Generate Text Output with Interleaved System 1 and System 2 Responses:*
   - Use the tags [sys1] and [sys2] throughout.
   - Aim for a lengthier, detailed response. Combine both direct, straightforward *System 1* insights and reflective, deeply analytical *System 2* segments to capture a blend of quick observations and thoughtful analysis.

---

### *Example Input Text:*

You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.

---

### *Processed Output with System 1 and System 2 Thinking:*

The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.

[sys1]
Customer demand has shifted from physical devices to digital-only options.
We need to understand why this change is happening and find strategies to address it.
Key factors may include cost, convenience, and technological trends.
[end sys1]

[sys2]
**2.1 Understand the Problem:**
The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.

*Reflection:* 
"I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"

*Creative Perspective:* 
"Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"

**2.2 Analyze the Information:**
There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.

*Reflection:* 
"Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"

*Creative Perspective:* 
"Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"

**2.3 Generate Hypotheses:**
1. Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
2. There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
3. Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
4. Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
5. Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
6. Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
7. Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
8. There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
9. Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
10. Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)

*Reflection:* 
"Have I considered all possible influences? Are there any surprising factors that could explain this shift?"

*Creative Perspective:* 
"Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"

**2.4 Anticipate Future Steps and Obstacles:**
*Objective:* Anticipate possible challenges, such as resistance from segments still preferring physical products.

*Reflection:* 
"What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"

*Creative Perspective:* 
"Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"

[end sys2]

[sys1]
To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions.
Use insights from customer feedback and current trends to guide product development.
Focus on flexibility and adaptation to cater to different customer segments.
[end sys1]


---
abstract: |
    Dead time affects many of the instruments used in X-ray astronomy, by producing a strong distortion in power density spectra. This can make it difficult to model the aperiodic variability of the source or look for quasi-periodic oscillations. Whereas in some instruments a simple a priori correction for dead-time-affected power spectra is possible, this is not the case for others such as [[*NuSTAR*]{}]{}, where the dead time is non-constant and long ($\sim$2.5ms). @Bachetti+15 suggested the cospectrum obtained from light curves of independent detectors within the same instrument as a possible way out, but this solution has always only been a partial one: the measured [$\mathrm{rms}$]{}was still affected by dead time, because the width of the power distribution of the cospectrum was modulated by dead time in a frequency-dependent way.

    In this Letter, we suggest a new, powerful method to normalize dead-time-affected cospectra and power density spectra. Our approach uses the difference of the Fourier amplitudes from two independent detectors to characterize and filter out the effect of dead time. This method is crucially important for the accurate modeling of periodograms derived from instruments affected by dead time on board current missions like [[*NuSTAR*]{}]{}and [[*Astrosat*]{}]{}, but also future missions such as [[*IXPE*]{}]{}.
author:
- Matteo Bachetti
- Daniela Huppenkothen
title: 'No time for dead time - Use the Fourier amplitude differences to normalize dead time-affected periodograms'
---

Introduction {#sec:intro}
============

Dead time is an unavoidable and common issue of photon-counting instruments. It is the time ${\ensuremath{t_d}\xspace}$ that the instrument takes to process an event and be ready for the next event. In most current astronomical photon-counting X-ray missions, dead time is of the *non-paralyzable* kind, meaning that the instrument does not accept new events during dead time, avoiding a complete lock of the instrument if the incident rate of photons is higher than $1/{\ensuremath{t_d}\xspace}$. Being roughly energy-independent, dead time is not usually an issue for spectroscopy, as it only affects the maximum rate of photons that can be recorded, so it basically only increases the observing time needed for high quality spectra.

For timing analysis, the effect of dead time is far more problematic. Dead time heavily distorts the periodogram, the most widely used statistical tool to estimate the power density spectrum (PDS)[^1], with a characteristic pattern that is stronger for brighter sources. It is often not possible to disentangle this power spectral distortion due to dead time and the broadband noise components characterizing the emission of accreting systems. In the special case where dead time is constant, its shape can be modeled precisely [@Zhang+95; @Vikhlinin+94]. However, dead time is often different on an event-to-event basis (e.g., in [[*NuSTAR*]{}]{}), and it is not obvious how to model it precisely, also because the information on dead time is often incomplete in the data files distributed by HEASARC [see, e.g. @Bachetti+15]. When using data from missions carrying two or more *identical and independent* detectors like [[*NuSTAR*]{}]{}, @Bachetti+15 proposed an approach to mitigate instrumental effects like dead time exploiting this redundancy: where in standard analysis, light curves of multiple detectors are summed before Fourier transforming the summed light curve, it is possible to instead Fourier-transform the signal of two independent detectors and combine the Fourier amplitudes in a *cospectrum* – the real part of the cross spectrum – instead of the periodogram. Since dead time is uncorrelated between the two detectors, the resulting powers have a mean white noise level fixed to 0, which resolves the first and most problematic issue created by dead time (see details in @Bachetti+15); however, the resulting powers no longer follow the statistical distribution expected for power spectra, and their probability distribution is frequency-dependent. Whereas a noise cospectrum in the absence of dead time would follow a Laplace distribution [@HuppenkothenBachetti18], dead time affects the width of the probability distribution for cospectral powers and modulates the measured [$\mathrm{rms}$]{}similarly to the distortion acted on power spectra. In this Letter, we show a method to precisely recover the shape of the periodogram by looking at the difference of the Fourier amplitudes of the light curves of two independent detectors. This difference, in fact, contains information on the uncorrelated noise produced by dead time, but not on the source-related signal which is correlated between the two detectors. This allows to disentangle the effects of dead time from those of the source variability.

In [Section ]{}[sec:fourierdiff]{} we show that, in the absence of dead time, the difference of the Fourier amplitudes calculated from two independent detectors contains the sum of the correlated signal (the source signal) and uncorrelated noise (detector-related noise), and that their difference eliminates the source part. In [Section ]{}[sec:fadsec]{} we use extensive simulations to show how to use this fact to correct dead-time-affected periodograms, and we describe the limitations of this method.

On the difference of Fourier amplitudes {#sec:fourierdiff}
=======================================

Let us consider two identical and independent detectors observing the same variable source, producing independent and strictly simultaneous time series, with identical even sampling $\delta t$, $\mathbf{x} = \{x_k\}_{k=1}^N$ and $\mathbf{y} = \{y_k\}_{k=1}^N$. For a stochastic process (e.g. $1/\nu$-type red noise), the Fourier amplitudes will vary as a function of $N_{\mathrm{phot}}P(\nu)/4$, where $P(\nu)$ (Leahy-normalized, @Leahy+83) is the shape of the power spectrum underlying the stochastic process, and $N_{\mathrm{phot}}$ denotes the number of photons in a light curve.

If the two detectors observe the same source simultaneously, the amplitudes and phases of the stochastic process will be shared among $\mathbf{x}$ and $\mathbf{y}$, while each light curve will be affected *independently* by the photon counting noise in the detector, as well as the dead time process.

Dead time can be considered a convolution on the signal [@Vikhlinin+94]. Following the convolution theorem the Fourier transform $\mathcal{F}$ of dead time-affected light curves will be the *product* of the Fourier transform of the signal $\mathcal{S}$ and the Fourier transform of the dead time filter $\mathcal{D}$: $$\mathcal{F}(\nu) = \mathcal{S}(\nu)\cdot\mathcal{D}(\nu)$$ For a large enough number of data points $N$, the complex Fourier amplitudes $S_j$ will be composed of a sum of two independent random normal variables for the intrinsic red noise variability and the detector photon counting noise, respectively: $S_j = S_{sj} + S_{nj}$, with $\Re (S_{sj}) \sim {\ensuremath{{\mathcal N}}\xspace}(0, \sigma_{sj}^2)$ and $\Re(S_{nj}) \sim {\ensuremath{{\mathcal N}}\xspace}(0, \sigma_n^2)$, and similarly for the imaginary parts. The red noise variance $\sigma_{sj}^2 = \sigma_{s}^2(\nu) = N_\mathrm{phot}P(\nu)/4$ (where $N_{\mathrm{phot}} = \sum_{k=1}^{N}{x_k}$) is given by the power spectrum of the underlying stochastic process and is frequency-dependent. However, the photon counting noise $\sigma_n^2 = N_\mathrm{phot}/2$ is independent of frequency. Note that $S_{xsj} = S_{ysj}$, because the amplitudes of the stationary noise process will be the same for the Fourier transforms of $\mathbf{x}$ and $\mathbf{y}$ for the case considered here, while the components due to white noise differ between the two time series. The dead time filter affects the sum of signal and white noise amplitudes as a multiplicative factor and only depends on count rate, which is equal for both light curves given identical detectors. Thus, the difference between the Fourier amplitudes for the two time series $\mathbf{x}$ and $\mathbf{y}$ will be: $$F_{xj} - F_{yj} = (S_{xj} - S_{yj})\cdot D_j = (S_{xnj} - S_{ynj})\cdot D_j$$ Because $S_{xsj} = S_{ysj}$, but $S_{xnj} \neq S_{ynj}$ (since the white noise component is formed in each detector separately), the *difference* of the real and imaginary Fourier amplitudes between the two light curves effectively encodes the white noise component only, multiplied by the Fourier transform of the dead time filter. This fact effectively allows us to separate out the (source-intrinsic) red noise from the spurious variability introduced by dead time: if we can extract the shape of the dead time filter $|\mathcal{D}|^2$ from the Fourier amplitude differences of the two detectors, we can use it to correct the shape of the periodogram. In the following section, we lay this procedure out in more detail, and describe its limits in Section \[sec:caveat\].

The FAD method {#sec:fadsec}
==============

Data simulation {#sec:data}
---------------

All simulated sets in this paper were produced and analyzed with a combination of the two Python libraries `stingray`[^2] [@huppenkothen2016] and `HENDRICS` v.`3.0b2` [formerly known as MaLTPyNT; @2015ascl.soft02021B], both affiliated to Astropy [@astropy2013; @astropy18].

We used the same procedure and algorithms described by @Bachetti+15, Section 4, which we briefly summarize here. We used the @timmer1995 method to create a red noise light curve starting from a given power spectral shape. This method is implemented in the `stingray.simulate` module. This step needs to be done carefully: if the initial light curves contain significant random noise, the process for the creation of events creates a random variate on the top of the local count rate–which is varying randomly already–producing a non-Poissonian final light curve. We initially simulated light curves with a very high mean “count rate” such that the Poisson noise was relatively small. We then renormalized the light curves to the wanted (lower) count rate and [$\mathrm{rms}$]{}and finally used these light curves to simulate event lists using rejection sampling, implemented in the `stingray.Eventlist.simulate_times()` method. Then, the `hendrics.fake.filter_for_deadtime()` function was used to apply a non-paralyzable dead time of 2.5ms to the simulated event lists. For more details on the simulated data sets and the validation of the simulation infrastracture, see also Section \[sec:correction\] and the available Jupyter notebooks[^3][^4]. After producing these synthetic event lists, we started the standard timing analysis: we produced light curves with a bin time of $\sim0.244$ms, and calculated power spectral products (cospectrum, periodogram) over segments of these light curves using `stingray`.

First test: white noise {#sec:wndeadtime}
-----------------------

As laid out in Section \[sec:fourierdiff\], the difference of Fourier amplitudes from two independent but identical detectors shows no source variability, but *still shows the same distortion* due to dead time (See Figure  \[fig:fourierdiff\], left panel, where this is shown with red noise). Let us simulate two constant 1000sec light curves with an incident mean count rate of 2000 counts/sec and a dead time of 2.5 ms, as we would expect from the two identical detectors of [[*NuSTAR*]{}]{}observing the same stable X-ray source. The Fourier amplitudes of the light curves from the two detectors are heavily distorted by dead time, with the characteristic damped oscillator-like shape [@Vikhlinin+94; @Zhang+95] (Figure \[fig:fourierdiff\], middle panel). Therefore, using the difference between the Fourier amplitudes in two detectors, we can in principle renormalize the periodogram so that only the source variability alters its otherwise flat shape.

As shown in Figure \[fig:fourierdiff\] (right panel), the single-detector Fourier amplitudes are proportional on average to the difference of the Fourier amplitudes in different realizations, with a constant factor $1/\sqrt{2}$. Therefore, we expect that the periodogram will be proportional to the square of the Fourier amplitude difference, divided by 2. Let us try to *divide the periodogram by a smoothed version of the squared Fourier differences*, and multiply by 2. For smoothing, we used a Gaussian running window with a window width of 50 bins. Given that the initial binning had 50 bins/Hz, this interpolation allows an aggressive smoothing over bins whose y value does not change significantly. We call this procedure the **Fourier Amplitude Difference** (hereafter FAD) **correction**.

The results of this correction are shown in Figure \[fig:comparison\]. Starting from a heavily distorted distribution of the powers, applying the FAD correction “flattens” remarkably well the white noise level of the periodogram and the distribution of the scatter of the white noise periodogram and cospectrum. Also, it reinstates a correct distribution of powers, following very closely the expected $\chi^2_2$ distribution (@Lewin+88; Figure \[fig:dist\], right). Analogously, the corrected cospectrum will follow the expected Laplace distribution (@HuppenkothenBachetti18; Figure \[fig:dist\], left). While the original dead time-affected cospectrum had a distorted, frequency-dependent [$\mathrm{rms}$]{}level, the FAD-corrected cospectrum returns to the correct distribution at all frequencies.

The FAD algorithm in detail {#sec:fad}
---------------------------

In practice, the FAD correction algorithm in a generic case would work as follows:

1.  split the light curves from two independent, identical detectors into segments as one would do to calculate standard averaged periodograms;

2.  for each pair of light curve segments:

    -   calculate the Fourier transform of each detector separately, and then of the summed detectors (hereafter total-intensity);

    -   save the unnormalized Fourier amplitudes;

    -   multiply these Fourier amplitudes by $\sqrt{2/N_{ph}}$ (that would give Leahy-normalized periodograms if squared);

    -   subtract the Leahy-normalized Fourier amplitudes of the two detectors between them, *take the absolute value*, and obtain this way the Fourier Amplitude Differences (FAD);

    -   *smooth* the FAD using a Gaussian-window interpolation with a large number of bins, in our case all the bins contained in 1-2 Hz, but this might need an adjustment at extreme count rates ($\gtrsim 10/{\ensuremath{t_d}\xspace}$), where significant gradients in the white noise periodogram can occur in less than 1 Hz;

    -   use the separated single-detector and total-intensity *unnormalized* Fourier amplitudes to calculate the periodograms (and/or the cospectrum);

    -   divide all periodograms (and/or the cospectrum) by the smoothed and squared FAD, and multiply by 2.

3.  normalize the periodograms to the wanted normalization (e.g. @Leahy+83 or fractional [$\mathrm{rms}$]{}: @BelloniHasinger90 [@Miyamoto+91]).

FAD correction of generic variable periodograms {#sec:correction}
-----------------------------------------------

We are now ready to verify whether the FAD-corrected periodogram is a good approximation to the dead time-free periodogram. To test this, we produced a number of different synthetic datasets as explained in Section \[sec:wndeadtime\], containing different combinations of QPOs and broadband noise components, expressed as pairs of Lorentzian curves[^5] We first calculated the periodogram and cospectrum of the dead time-free data, averaged over 64–512s intervals. Then, we applied a 2.5ms dead time filter to the event list and applied the FAD correction, as described in Section \[sec:fad\]

All spectra were then expressed in fractional [$\mathrm{rms}$]{}normalization [@Miyamoto+91; @BelloniHasinger90]. In this normalization, the integrated model returns the full fractional [$\mathrm{rms}$]{}of the light curve, and the dead time-free and the FAD-corrected periodograms should be consistent over the full frequency range. An example of this analysis is shown in Figure \[fig:pds\]: the FAD successfully corrects so well periodograms and cospectra when compared to dead-time free simulated spectra, that in the figure they are almost indistinguishable.

We ran extensive simulations testing how the method performs (1) for a range of different input count rates, leading to dead-time effects of different magnitude in the output periodograms and cospectra, and (2) when the light curves do not have the same count rate (since detectors may in reality have slightly different efficiencies). We fitted all spectra with a two-Lorentzian model, plus a constant offset to account for the white noise level in periodograms. We calculated the [$\mathrm{rms}$]{}by integrating the model fitted above over the full frequency range, and compared the results in the dead-time-free and FAD-corrected cases.

Simulation results and discussion {#sec:caveat}
---------------------------------

The simulations described above show that the shape of the periodogram is precisely corrected by the FAD procedure if *the input light curves have the same count rate* and *for values of the input count rate and [$\mathrm{rms}$]{}that are not too extreme*. Differing input count rates in different detectors matter in practice only for the single-detector periodogram, but *not* for cospectra and total-intensity periodograms. At high count rates, single-detector periodograms are corrected very well only if the two detectors have very similar count rates, and count rates must be more similar at higher count rates in order for the correction to apply. However, we find that the total-intensity periodogram and the cospectrum remain well corrected by the FAD even if the count rate in the two detectors differs by 30% in most cases. Therefore, we recommend to use the FAD very carefully with single-detector periodograms, which should not be an issue given that the total-intensity periodogram is more sensitive and more convenient to use. A comparison with the cospectrum, which is not affected by white noise level distortions, is always recommended.

However, we find that the FAD correction consistently overestimates the integrated [$\mathrm{rms}$]{}when the count rate and [$\mathrm{rms}$]{}are *both* very high, *in particular at low frequency* (See Figure \[fig:errors\]). At $\sim$200 ct/s and 50% [$\mathrm{rms}$]{}, the relative overestimation is below 5% (meaning that if the true [$\mathrm{rms}$]{}is 50%, the measured [$\mathrm{rms}$]{}is between 50 and 53%) and it is symmetrically distributed around 0, as expected from statistical uncertainties. At higher incident rates and [$\mathrm{rms}$]{}, the uncertainty distribution is biased towards positive relative errors, implying an overestimation of the [$\mathrm{rms}$]{}. This should not be a problem in most use cases, when the [$\mathrm{rms}$]{}is used as a rough indicator for spectral state. If very precise measurements of [$\mathrm{rms}$]{}are needed (for example, to calculate [$\mathrm{rms}$]{}/energy spectra), it is safer to account for this bias through simulations. As a rough rule-of-thumb, the bias in the measured fractional [$\mathrm{rms}$]{}increases *linearly* with the count rate and *quadratically* with [$\mathrm{rms}$]{}. A practical way to estimate this effect during analysis is to apply the FAD, obtain a best fit model, calculate the [$\mathrm{rms}$]{}, and simulate a number of realizations of the light curve to evaluate the amount of overestimation involved[^6].

Conclusions
===========

In this Letter we described a method to correct the normalization of dead time-affected periodograms. This method is valid in principle for 1) correcting the shape of the periodogram, eliminating the well known pattern produced by dead time, and 2) adjusting the white noise standard deviation of periodogram and cospectra to its correct value at all frequencies. In general, we recommend applying the FAD correction to both the periodogram and the cospectrum. The periodogram, if obtained by the sum of the light curves, can yield a higher signal-to-noise ratio. However, the white noise level subtraction is not always very precise due to mismatches in the mean count rate in the two light curves. A comparison with the FAD-corrected cospectrum, to verify visually the white noise subtraction, is always recommended: the white noise subtraction is the most important step when calculating the significance of a given feature in the periodogram [e.g. @Barret+12; @Huppenkothen+17]. The cospectrum has indeed the advantage of not requiring white noise level subtraction.

We performed a number of simulations to test the validity of our method and explore its performance in the limits of high count rates as well as detectors with mis-matched efficiencies. In all cases, we find that the adjustment of the white noise standard deviation in the periodogram and the cospectrum works remarkably well, allowing to make a confident analysis of X-ray variability even in sources where this was precluded until now. Only in cases where the count rate and the [$\mathrm{rms}$]{}are *both* very high ($>$500 ct/s incident, $>$40% resp.), the FAD correction leads to an overestimation the [$\mathrm{rms}$]{}, even if the white noise level of the periodogram remains flat.

We thank the anonymous referee for providing very insightful feedback. We thank David W. Hogg for useful discussions on the topic of Fourier analysis, and Jeff Scargle for useful suggestions and comments. MB is supported in part by the Italian Space Agency through agreement ASI-INAF n.2017-12-H.0 and ASI-INFN agreement n.2017-13-H.0. DH is supported by the James Arthur Postdoctoral Fellowship and the Moore-Sloan Data Science Environment at New York University. DH acknowledges support from the DIRAC Institute in the Department of Astronomy at the University of Washington. The DIRAC Institute is supported through generous gifts from the Charles and Lisa Simonyi Fund for Arts and Sciences, and the Washington Research Foundation.

\[1\][[\#1](#1)]{}

, [et al.]{} 2013, , 558, A33

Astropy Project, [et al.]{} 2018, arXiv, arXiv:1801.02634

Bachetti, M. 2015, MaLTPyNT: Quick look timing analysis for NuSTAR data, Astrophysics Source Code Library, ascl:1502.021

Bachetti, M., Harrison, F. A., Cook, R., [et al.]{} 2015, ApJ, 800, 109

Barret, D., & Vaughan, S. 2012, ApJ, 746, 131

Belloni, T., & Hasinger, G. 1990, A[&]{}A, 230, 103

Hunter, J. D. 2007, Computing in Science and Engineering, 9, 90

Huppenkothen, D., & Bachetti, M. 2017, accepted to ApJ, arXiv:1709.09666

, D., [Bachetti]{}, M., [Stevens]{}, A. L., [Migliari]{}, S., & [Balm]{}, P. 2016, [Stingray: Spectral-timing software]{}, Astrophysics Source Code Library, , , ascl:1608.001

Huppenkothen, D., Younes, G., Ingram, A., [et al.]{} 2017, ApJ, 834, 90

Kluyver, T., Ragan-Kelley, B., P[é]{}rez, F., [et al.]{} 2016, in “Positioning and Power in Academic Publishing: Players, Agents and Agendas”, 87–90

Leahy, D. A., Darbro, W., Elsner, R. F., [et al.]{} 1983, ApJ, 266, 160

Lewin, W. H. G., van Paradijs, J., & van der Klis, M. 1988, SSRv, 46, 273

Miyamoto, S., Kimura, K., Kitamoto, S., Dotani, T., & Ebisawa, K. 1991, ApJ, 383, 784

, J., & [Koenig]{}, M. 1995, , 300, 707

van der Walt, S., Colbert, S. C., & Varoquaux, G. 2011, Computing in Science Engineering, 13, 22

Vikhlinin, A., Churazov, E., & Gilfanov, M. 1994, A&A, 287, 73

Zhang, W., Jahoda, K., Swank, J. H., Morgan, E. H., & Giles, A. B. 1995, ApJ, 449, 930

[^1]: here we will use the term PDS for the actual source power spectrum, and *periodogram* to indicate our estimate of it, or otherwise said, the realization of the “real” power spectrum we observe in the data

[^2]: The library is under heavy development. For this work we used the version identified by the hash `3e64f3d`. See <https://github.com/StingraySoftware/notebooks/> for tutorials on simulations, light curve production and timing analysis with Stingray

[^3]: <https://github.com/matteobachetti/deadtime-paper-II/>

[^4]: <https://github.com/StingraySoftware/HENDRICS/tree/master/notebooks>

[^5]: See the notebooks at <https://github.com/matteobachetti/deadtime-paper-II> to reproduce the analysis plotted in the Figures of this paper and more. The algorithm described in Section \[sec:fad\] is contained in the `fad_correction.py` file in the notebooks directory.

[^6]: relevant code can be found in Jupyter notebooks in the following GitHub repository: <https://github.com/matteobachetti/deadtime-paper-II>