<figure><img src="image_1.png" /></figure>

HAL
open science

# Radiation Hardness Assurance Through System-Level Testing: Risk Acceptance, Facility Requirements, Test Methodology, and Data Exploitation

Andrea Coronetti, Ruben Garcia Alia, Jan Budroweit, Tomasz Rajkowski, Israel Da Costa Lopes, Kimmo Niskanen, Daniel Soderstrom, Carlo Cazzaniga, Rudy Ferraro, Salvatore Danzeca, et al.

<figure><img src="image_2.png" /></figure>

## To cite this version:

Andrea Coronetti, Ruben Garcia Alia, Jan Budroweit, Tomasz Rajkowski, Israel Da Costa Lopes, et al.: Radiation Hardness Assurance Through System-Level Testing: Risk Acceptance, Facility Requirements, Test Methodology, and Data Exploitation. IEEE Transactions on Nuclear Science, 2021, 68 (5), pp.958-969. 10.1109/TNS.2021.3061197. hal-03341566

HAL Id: hal-03341566

https://hal.science/hal-03341566

Submitted on 9 Jun 2022

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Radiation Hardness Assurance Through System-Level Testing: Risk Acceptance, Facility Requirements, Test Methodology, and Data Exploitation

Andrea Coronetti<sup>1,2</sup>, Student Member, IEEE, Rubén García Alía<sup>1,2</sup>, Member, IEEE, Jan Budroweit<sup>1</sup>, Associate Member, IEEE, Tomasz Rajkowski<sup>1,2</sup>, Member, IEEE, Israel Da Costa Lopes<sup>1</sup>, Kimmo Niskanen<sup>1,2</sup>, Member, IEEE, Daniel Söderström<sup>1,2</sup>, Student Member, IEEE, Carlo Cazzaniga<sup>1,2</sup>, Rudy Ferraro<sup>1</sup>, Salvatore Danzeca, Julien Mekki<sup>1,2</sup>, Florent Manni, David Dangla, Cedric Virmontois, Nourdine Kerboub<sup>1</sup>, Graduate Student Member, IEEE, Alexander Koelpin<sup>1,2</sup>, Senior Member, IEEE, Frédéric Saigné, Pierre Wang, Member, IEEE, Vincent Pouget, Member, IEEE, Antoine Touboul<sup>1,2</sup>, Arto Javanainen<sup>1,2</sup>, Member, IEEE, Heikki Kettunen, Member, IEEE, and Rosine Coq Germanicus<sup>1</sup>

**Abstract**—Functional verification schemes at a level different from component-level testing are emerging as a cost-effective tool for those space systems for which the risk associated with a lower level of assurance can be accepted. Despite the promising potential, system-level radiation testing can be applied to the functional verification of systems under restricted intrinsic boundaries. Most of them are related to the use of hadrons as opposed to heavy ions. Hadrons are preferred for the irradiation of any

Manuscript received January 20, 2021; revised February 12, 2021 and February 15, 2021; accepted February 18, 2021. Date of publication February 22, 2021; date of current version May 20, 2021. This work was supported by the European Union's Horizon 2020 Research and Innovation Program through the Marie Skłodowska Curie (MSC) under Grant 721624.

Andrea Coronetti is with CERN, 1211 Geneva, Switzerland, and also with the Department of Physics, University of Jyväskylä, 40014 Jyväskylä, Finland (e-mail: andrea.coronetti@cern.ch).

Rubén García Alía, Rudy Ferraro, and Salvatore Danzeca are with CERN, 1211 Geneva, Switzerland.

Jan Budroweit is with DLR, 28359 Bremen, Germany.

Tomasz Rajkowski and Pierre Wang are with 3D Plus, 78530 Buc, France.

Israel Da Costa Lopes, Kimmo Niskanen, Frédéric Saigné, Vincent Pouget, and Antoine Touboul are with the Institute d'Électronique et des Systèmes, Université de Montpellier, 34090 Montpellier, France.

Daniel Söderström and Heikki Kettunen are with the Department of Physics, University of Jyväskylä, 40014 Jyväskylä, Finland.

Carlo Cazzaniga is with the Science and Technology Facilities Council, Didcot OX11 0QX, U.K.

Julien Mekki, Florent Manni, David Dangla, and Cedric Virmontois are with CNES, 31400 Toulouse, France.

Nourdine Kerboub is with CERN, 1211 Geneva, Switzerland, and also with CNES, 31400 Toulouse, France.

Alexander Koelpin is with the Institut für Hochfrequenztechnik, TUHH, 21073 Hamburg, Germany.

Arto Javanainen is with the Department of Physics, University of Jyväskylä, 40014 Jyväskylä, Finland, and also with the Department of Electrical Engineering and Computer Science, Vanderbilt University, Nashville, TN 37235 USA.

Rosine Coq Germanicus is with Crismat, Iut, Unicaen, Ensicaen, CNRS, Normandie University, 14000 Caen, France.

Color versions of one or more figures in this article are available at https://doi.org/10.1109/TNS.2021.3061197.

Digital Object Identifier 10.1109/TNS.2021.3061197

bulky system, in general, because of their deeper penetration capabilities. General guidelines about the test preparation and procedure for a high-level radiation test are provided to allow understanding which information can be extracted from these kinds of functional verification schemes in order to compare them with the reliability and availability requirements. The use of a general scaling factor for the observed high-level cross sections allows converting test cross sections into orbit rates.

**Index Terms**—Commercial off-the-shelf (COTS), facilities, neutrons, protons, radiation hardness assurance, risk acceptance, single-event effect (SEE), small satellites, system-level testing, test methodology, total ionizing dose (TID).

## I. INTRODUCTION

COMMERCIAL OFF-THE-SHELF (COTS) devices have been gaining popularity within the radiation community during the last two decades, thanks to their higher electrical and electronic performance, when compared to similar rad-hard parts, and to their reduced price and lead time. Similarly, interest has been growing around highly integrated solutions manufactured within the same package (e.g., system-on-chip, SoC) or assemblies of discrete devices and integrated circuits (ICs) on printed circuit boards (PCBs), boxes, or modules.

The radiation testing single-event effect (SEE) [1], [2] and total ionizing dose (TID) [3] standards developed by the community are in a continuous struggle when it comes to keeping up with the innovation introduced by brand new devices (e.g., flip-chips, multiple chips stacked within the same package, 3-D layouts) which outperform those devices the standards were tailored for. Among the main criticalities stands the necessity of making the sensitive volumes (SVs) of the devices and ICs accessible to those beams, such as heavy ions, which are typically characterized by high linear energy
transfer (LET), but short range in matter. It is noted that for some of these layouts, decapsulation may be unachievable in some cases.

In view of the emerging challenges, the radiation community started questioning whether it was possible to perform qualification of devices and ICs that could overcome the usual inconveniences associated with standard testing (e.g., use of vacuum chambers, decapsulation) by using deeper penetrating beams, such as high-energy protons, as a proxy for heavy ions. An effort that started more than two decades ago [4], [5] and whose potentialities and limitations are summarized in a book-of-knowledge for proton board-level irradiation [6], [7].

Among the potentialities stands the verification of the soft error response of an entire set of devices at a reduced cost. At the same time, very loose bounds can be applied to hard and destructive SEEs (DSEEs) coverage without heavy-ion testing [8], [9] and, likely, the information extracted is not sufficient to perform a rigorous TID worst case analysis (WCA) for the considered devices [10].

Nevertheless, due to its cost competitiveness, system-level testing may find wider applications when it comes to space missions associated with higher risk acceptance, for example, CubeSats and NanoSats. Due to schedule and cost constraints, such space missions may not afford the cumbersome qualification based on component-level testing and the mentioned standards, often running into the highly disputable no-testing approach. For such missions, system-level testing on either a radiation model or the flight model itself may provide a higher level of confidence on the mission success likelihood while being compatible with schedule and cost restrictions.

Approaches based on system-level testing are nowadays already in use for terrestrial applications, for which neutrons can provide a sufficient coverage for DSEEs, and TID degradation is not an issue. In the accelerator field [11], system-level testing is used in a complementary fashion with respect to component-level testing in that it is used as a qualification tool only for those devices which are not critical within the design of the system and as a final verification of the system functionality.

A few examples for space applications have been reported in the literature by CNES [12], [13], DLR [14], and University of Montpellier [15]. System-level radiation testing may find wider applications in the future for space missions having criticality classes Q1 and Q2 as defined in the European Space Agency (ESA) COTS initiative review [16].

Under such promises, this work aims at synthesizing guidelines on how to perform system-level radiation testing with hadrons as a verification tool for high-risk acceptance space missions. This will include providing a common language among actors in system design, development, and verification, a guidance among the various criteria to be borne in mind in order to decide whether to go for system-level testing (and under which conditions), the best suited facilities to perform system-level testing, the test logic and procedure to follow, as well as the usage of the high-level data extracted from the test.

## II. "SYSTEM" AND SYSTEM-LEVEL RADIATION TESTING

A component can be defined as any electronic device which cannot be physically partitioned without affecting its capability of delivering the intended functionality [17]. In this context, anything that is manufactured on a single chip has to be considered as a component, for example, SoC.

If the terminology "system" is applied to everything else at a higher integration level than a component, then a system can be anything from a PCB, with a few discrete components, to a whole satellite. In terms of exposing such assemblies to a radiation field, the challenges are somewhat similar over this full scale (i.e., not easy to ensure uniform irradiation with heavy ions, not easy to access all the SVs by decapsulation once all devices are placed within the system layout, not easy to perform standard TID testing). In addition, any assembly of two or more devices can lead to the generation of radiation-chain effects, that is, malfunctioning of a device which is caused from a radiation effect occurring in another device, which is feeding signals or information to the device in which the malfunctioning is observed (e.g., data corruption in a memory fed to a microprocessor).

Systems can be classified according to the following categories:

1) custom-designed based on COTS, graded, or rad-hard components;

2) modified off-the-shelf (MOTS) systems;

3) fully commercial.

Custom-designed systems based on COTS are built in-house by the satellite designer/integrator. In this case, the developer has control over part screening and selection, traceability of components, architecture, and can include radiation effects tolerance and mitigation within the system design. A typical example is the Function Generator Controller Lite (FGCLite) system for the CERN large hadron collider [18].

Fully commercial systems are manufactured by a third party (likewise components) and are intended to be used as they are. Similar to COTS devices, the satellite designer/integrator may not be provided with more information than those listed in the data sheet. Thus, part selection and traceability as well as the system architecture are often not available.

MOTS systems are an intermediate category. They are commercial systems whose radiation tolerance is improved by the end-user, thanks to collaboration agreements with the manufacturer allowing access to information related to the internal architecture and the Bill-of-Materials (BoM) to either apply mitigations or part replacement.

In general, a satellite can be thought of as a custom-designed system. However, this does not exclude some of its subsystems from being based on MOTS or fully commercial solutions. The radiation tolerance of custom-designed systems in space is generally attained through component-level testing and screening. At the other end of the spectrum, the radiation response of fully commercial systems can be established only though system-level testing, with the only alternative of a cumbersome reverse engineering process.

System-level radiation testing consists in the experimental verification of the compliance of the system to the reliability
<figure><img src="image_4.png" /><figcaption>Fig. 1. System-level radiation testing with respect to risk acceptance and cost.</figcaption></figure>

and availability requirements defined for the mission by oper-
ating the finished system under radiation. The extent to which
the previous definition can be applied depends on whether
the system-level test can provide the necessary insight to the
radiation performances.

System-level radiation testing can be, in general, used to
shed light upon:

1) functional reliability with the existing set of components and architecture;

2) functional availability with the embedded software and firmware and for the selected space environment;

3) criticalities arising from the radiation effects of single components or dependent faults and failures;

4) criticalities due to the design itself;

5) whether the system can perform self-recovery;

6) whether other implemented mitigation techniques (e.g., derating, transient filtering, error correction codes) are effective;

7) additional system-level mitigations to be implemented at hardware or software level.

Other than a verification tool for high-risk acceptance space
missions that would alternatively follow a no-testing scheme,
system-level testing can also be considered as a complement
to component-level characterization for noncritical subsystems
and as a final verification tool for very complex systems whose
components previously sustained a complete component-level
screening. For the latter, it is noted that the radiation response
of complex systems in the working configuration (although
they are designed following a rigorous qualification) may still
be dominated by dependent faults and failures or synergistic
effects that were not easy to anticipate by modeling (e.g.,
fault-tree analysis or failure mechanism effects and analysis).
In this case, system-level testing can exercise the system as
a stochastic fault injection tool in accordance with the actual
probability of fault occurrence. Fig. 1 summarizes some of
these concepts by positioning the various available options in
terms of risk acceptance and costs.

III. RISK ACCEPTANCE

When facing the decision on whether to go for system-level
radiation testing, the user shall carefully assess what kind
of coverage is achievable through system testing and what
are its limitations. NASA provides recommendations on this
subject [19] while accounting for mission environment, appli-
cation, and lifetime.

The mission environment plays a significant role in defining
the risk acceptance. The two main threats to the mission
reliability and success are 1) cumulative TID and total non-
ionizing dose (TNID) effects and 2) the stochastic DSEEs
related to a single particle strike. Note that not all DSEEs
are stimulated by hadrons (due to the limited energy imparted
to the secondary ions [20]), whereas they can be stimulated by
heavy ions provided that the LET is high enough. The reasons
why these two threats may be very critical when it comes to
system-level radiation testing are that:

1) this test cannot provide the wide insight necessary to perform WCA following parametric drifts induced by TID and TNID;

2) the use of hadrons does not cover the full spectra of particles encountered in the space environment and responsible for DSEEs.

These radiation effects are not only critical when it comes to determining and verifying the radiation response, but have implications on the design of the system itself by, for example, derating [21] of the components in order to avoid DSEEs. That is why whenever components susceptible to DSEEs [such as single-event burnout (SEB) or single-event gate rupture (SEGR)] have to be used in the system, a preliminary characterization at component level is always mandatory. This is in order to establish the correct derating to apply to the system. Use of default derating factors, for example, 50% in the case of aviation [22] may not provide sufficient coverage due to the presence of heavy ions in the space environment. The costs associated with a change in the design upon discovery of a failure at a late stage in the development, as it would be for verification by system-level testing only, would overpass the initially predicted cost benefits of the verification [23].

The radiation response variability over different technolo-
gies and semiconductor materials is also an important factor
to consider because of several subtle radiation effects that may
characterize certain devices. One of them is the enhanced low-
dose-rate sensitivity (ELDRS) [24], the effect of which is to
produce a larger degradation when the dose rate is lower (like
in the application) than usually applied in accelerated testing.
Component-level standards [3] provide recommendations for
testing devices that may be susceptible to ELDRS. However,
if the system-level radiation testing is performed through a
single verification in a hadronic environment (i.e., an environ-
ment obtained by nuclear spallation of a high-energy proton
beam with a high-Z target and usually composed of a wide
spectra of protons, neutrons, and pions or a selection of them), the dose rate may represent a lower constraint than the suitable hadron flux for the SEE screening. Thus, ELDRS may end up being untested and unassessed. At the same time, ELDRS does not usually appear below a TID of 10 krad(Si) [25]. Hence, there are space missions for which it can be neglected.

Variability in the degradation among materials may be a big deal when it comes to displacement damage (DD) [26] where variability cannot only be observed in how the materials degrade, but also in the different effects produced by different particles, for example, protons as opposed to neutrons, making the non-ionizing energy loss (NIEL) approximation fall apart. Once again, these effects are likely to happen when significant TNID has to be delivered to the devices, which may not be the case for most of the systems whose reliability could be verified through system-level radiation testing.

Similar to component-level radiation testing, the outcome of a system-level radiation test is described on a pass/fail basis when it comes to reliability. The main difference is the severity that a “fail” outcome has on the system design choices. Discarding a component through prescreening comes mainly at the cost of the beam time and test preparation. A fail outcome for the entire system may, in the best case, lead to a reiteration in the design in order to solve the issue encountered during the test by mitigation and, in the worst case, may require a full redesign of an already developed prototype.

Observing only high-level radiation effects on the system without delving deeper into the component characterization may also be problematic when it comes to implement solutions that could mitigate or solve a potential source of unreliability emerging as an outcome of the test. The failure of a device in the system makes it quite easy to identify the culprit. However, there may be other failures for which clear root causes may be hard to spot, for example, when failures are caused by the concurrent degradation of several devices or by other SEE-related dependent failures. Hence, also the depth of observability in system-level radiation testing may need to be calibrated in order to increase the number of observable parameters for a correct outcome interpretation. Clearly, this may lead to longer test-bench preparations and potential compatibility issues with the facility.

Generally, a “pass” outcome from system-level radiation testing comes with a limited level of confidence. The aforementioned component-level testing standards for TID suggest performing tests over 10 parts to assess and account for the intralot variability. The outcome of a single system-level test may not be replicable over other units (even when the traceability of the single components is respected) due to:

1) one or more units may fail due to unlikely radiation effects that cannot be reproduced on all units with the targeted fluence;

2) the units may be tested under different conditions of voltage, frequency, temperature, and application, which can impact both cumulative degradation and stochastic event probability [27].

For TID and TNID, the only possibility to increase the confidence on the outcome would be to test up to margined

<table><caption>TABLE I
CRITICALITY CLASSES TO DEFINE SYSTEM-LEVEL EFFECTS</caption><thead><tr><td>Class</td><td>Impact on the system</td><td>Action</td><td>Radiation effect naming</td></tr></thead><tbody><tr><td>0</td><td>Transparent to the system functionality</td><td>No action needed</td><td>-</td></tr><tr><td>1</td><td>Temporary impact on functionality</td><td>No action or simple mitigation through existing equipment</td><td>Soft loss of functionality</td></tr><tr><td>2</td><td>Availability impacted, but no mission loss</td><td>Supervisory circuitry added to have only temporary impact</td><td>Hard loss of functionality</td></tr><tr><td>3</td><td>Mission reliability not achieved</td><td>May require intervention on the system design and parts</td><td>Permanent loss of functionality</td></tr></tbody></table>

doses, whereas for SEEs, as long as the irradiation source provides negligible levels of TID and TNID, testing the flight model may mitigate the associated risk (although testing flight equipment shall also be carefully assessed and traded based on the risk of suffering DSEEs during the verification test itself).

## IV. RADIATION EFFECTS AT SYSTEM-LEVEL AND RADIATION-TOLERANT SYSTEM

Similar to component-level effects, a common language for system-level radiation effects, may be introduced to facilitate information exchange and data portability. The aim is to describe the radiation effects so that they are strictly connected to the functional reliability and availability of the system (hence, more promptly linked to the system requirements).

The main system-level radiation effect is the loss of functionality, that is, the condition under which the system stops either temporarily or permanently to deliver its intended top-level functionality or starts delivering it outside of specifications. A classification can be made based on the criticality chart in Table I.

Class 0 effects are radiation effects occurring on a device that do not propagate up to the top-level functionality because they are either filtered, masked, or unasserted at the moment they happen. All such effects will not have any impact on the system availability or reliability. Class 0 effects are generally attained once system mitigation is implemented.

Class 1 effects can produce visible, though very mild, effects on the system functionality, which in turn may affect the system availability. They are usually originated at component level by either single-event upsets (SEUs), single-event transients (SETs), multiple-cell upsets (MCUs), or single-event functional interrupts (SEFIs). Their limited impact is due to the fact that they may last one iteration in a digital processing system or that they can last for a few fractions of a second as analog signals. Note that not all SEFIs can be included in this category. SEFIs will be classified as soft losses of functionality (SLF) only if the system can recover from them without relying on power cycling.

Class 2 effects differ from class 1 because they may have a stronger impact, mainly due to the fact that the associated downtime for the system is longer. The larger downtime is usually associated with the need of performing a power cycle
of the whole system in order to remove the undesired radiation effect. This is the case of single-event latchups (SELs) and SEFIs. Note that both these events require the use of supervising circuitry to be removed. For the SEL, to detect the current increase and, for the SEFI, to detect the interruption of the function. Unless mitigated (by, e.g., scrubbing), some SEUs in the configuration logic and memory of digital devices may lead to continuous malfunctioning of the system that may not result in an SEFI. In this case, probably a power cycling will be needed as well to remove these uncorrectable SEUs. Hence, they may also be considered as hard losses of functionality (HLF).

Class 3 effects can affect the reliability of the system. Class 3 effects can have as root causes the degradation induced by TID and TNID, DSEEs, unmitigated milder SEEs, or even dependent failures. This includes SEBs, SEGRs, unprotected SELs, and unrecoverable SEFIs. In any case, the end effect observed during the test or the mission is that the system functionality is lost and cannot be recovered by any means. Note that this does not apply only to sudden failures due to stochastic events, but it also applies to continuous degradation imparted by TID and TNID. That is, the system may still be operating, but outside of the specifications that are set in the requirements (e.g., system unable to provide high enough voltage, provided output signals with too low margin with respect to noise).

The proposed classification is meant to provide a common taxonomy among users. However, other classifications [28] can be considered given their complementarity.

Other than the various degrees of loss of functionality, several degraded modes can be observed during a test. The simplest is the degradation of performance due to the parametric degradation from TID and TNID. That is, the system keeps on providing its full functionality, but nominally, this is provided at lower speed, with the system working at higher temperature or with the system requiring a higher power consumption. As long as the variations are still within the specifications, these effects are not to be considered losses, but just degradations.

Other degraded modes may be more impactful, but still not bring to end effects comparable to a loss of functionality. For instance, a few stuck bits in a memory device may reduce the total throughput of the memory, but this may be tolerable because the system does not make full use of the memory resources available. Even the failure of a single device due to SEB can be considered only a degradation if either the impact to the global functionality is limited (because the system used several of those same devices to accomplish its duty) or if there are redundancies.

Based on the pure top-level radiation effects, a radiation-tolerant system is a system that can provide its functionality under the declared specifications in the defined radiation environment while not suffering from permanent loss of functionality (PLF), that is, it is compliant with the reliability requirements. A radiation-tolerant system may suffer from HLF and SLF and manifest degraded modes of operation as long as their impact is compliant with the availability requirements.

<figure><img src="image_5.png" /><figcaption>Fig. 2. Range and LET of heavy ions at various energies that are within the current ground test capabilities.</figcaption></figure>

## V. BEAM AND FACILITY REQUIREMENTS FOR SYSTEM-LEVEL RADIATION TESTING

Concerning the most suitable beam conditions, some requirements are provided based on the cases for which the system under consideration can be as complex as a small satellite having a 3-D layout arrangement resulting in a volume of $50 \times 50 \times 50 \text{ cm}^3$. The considerations made and beam requirements proposed for such kind of system can be relaxed depending on the geometry and layout configuration of the system the user is willing to test.

Since the space environment cannot be reproduced with fidelity at ground-level accelerators (in terms of the full spectra of particles involved as well as dose rate over proton and heavy-ion fluxes ratio), some compromises have to be made in order to propose a system-level radiation testing methodology which can be applicable to the existing facilities.

The main driver for facility selection is the beam homogeneity, both depth-wise and over a wide enough surface. In order to ensure uniformity of the irradiation of the system, what standards typically require is that the homogeneity is kept within ±10%.

Homogeneous depth-wise irradiation is quite critical, as it can be ensured only by highly energetic and highly penetrating beams. In addition, the selected beam shall not be prone to strong fragmentation while traversing various layers of material at the penalty of decreasing the beam intensity and significantly altering its composition.

Fig. 2 reports the main features of protons and ions available at ground-level facilities (considering Europe and North America, although the whole state space is achieved only at NASA Space Radiation Laboratory (NSRL) [29]) nowadays based on the ion species and the primary energy. The LETs and ranges reported in the figure were calculated with Stopping and Range of Ions in Matter (SRIM) [30] using silicon as reference material. Note that if the range is used as a metric to qualify which ions may be suitable, no ions can be found that have both a surface LET of 30 MeV/(mg/cm²) and a penetration in silicon of 5 mm. This means that it is not very likely to find ions suitable for the radiation testing of systems having an even not so deep volume. Suitable ions in terms of range would have
LETs which can indeed be found in hadron secondary ions in silicon.

Note that considering only range and surface LET of the ions is not sufficient. Indeed, the figure does not provide any information about:

1) the ion beam fragmentation as the ions traverse the various layers of material, which results in a high-intensity reduction of the primary beam;

2) the fact that even for a single one-sided PCB, the various SVs may be under diverse thicknesses of packaging and shielding material, resulting in a nonhomogeneity of the surface LET at the SV [31];

3) the SVs of the devices may be shallower or deeper, resulting in further variable energy deposition event probability.

As a result, when using ions to irradiate a system, the devices composing the system will be subjected to spectra of particles, whose LET, Bragg peak distribution, and local flux will not guarantee homogeneity.

In terms of depth-wise homogeneity, high-energy protons are a better fit for energies of a few hundreds of MeV. In this case, it is not the primary LET of the proton that matters, rather it is the LET of the secondary ions generated by hadron-silicon interactions, which can be in the range 0-15 MeV/(mg/cm²). In addition, these ions are generated within the SV itself or in its close proximities. Protons do well for soft SEE testing; however, the initial LET of the secondary ions can often be insufficient to trigger those events requiring deep ionization tracks such as SEL, SEB, and SEGR [8], [9] because they would result in lower energy deposition events than with shallower volumes. That is why DSEE coverage cannot be ensured with proton testing.

Other than mono-energetic protons, depth-wise homogeneity can be ensured by beams with similar characteristics. This is the case of neutrons produced through nuclear spallation and of mixed fields made of protons, neutrons, and pions. This is, for instance, the case of the ChipIr [32] and the CERN Highly-Accelerated Mixed-field (CHARM) [33] facilities. Both facilities are characterized by spectra of particles from very low energies to very high energies (up to 800 MeV at ChipIr and up to 24 GeV at CHARM).

Fig. 3 provides a comparison of penetration capabilities of the hadronic beams available in a few selected facilities with respect to the penetration capabilities of the low-Earth orbit (LEO) proton environment. The LEO proton environment was calculated with the Cosmic-Ray Environment and Effects Models (CREME) online tool [34]. It accounts for both trapped proton and galactic cosmic proton spectra determined for an orbit with 800-km altitude, 98° inclination, and solar minimum conditions. Note that for the considered CHARM spectrum in the figure, the maximum hadron energy is 3 GeV. Also, for both the LEO environment and the facilities, the plotted beam intensity accounts only for hadrons having a minimum energy of 20 MeV.

The figure is meant to compare the penetration capabilities of the various fields into growing thickness of aluminum, which is taken as a representative of the diverse shielding provided by the various layers of materials of a typical space

system. The LEO proton spectrum decays very fast. After 2 cm of aluminum, its intensity is reduced to just 40% of that of the original. This is mainly due to the limited energy of trapped protons, making the most part of the environment. The three facility beams are not as strongly affected. After 2 cm of aluminum, the 200-MeV proton and ChipIr field will preserve more than 95% of the intensity of the primary beam, whereas CHARM is above 90%.

Hence, when irradiating a system in these facilities, and provided there is no more than 2 cm of aluminum of equivalent material between the front face and the back face, the beam intensity would reduce by less than 10%, ensuring depth-wise homogeneity. For 200-MeV protons and ChipIr that homogeneity would be maintained for up to 4 cm of equivalent aluminum.

Other than depth-wise homogeneity, beams produced by nuclear spallation are widely emitted in every direction. Hence, at sufficient distance from the source, the resulting beam will be homogeneous over a large surface. This is quantified in $70 \times 70 \text{ cm}^2$ for ChipIr and $100 \times 100 \text{ cm}^2$ for CHARM. Mono-energetic proton beams are usually developed to irradiate devices and are associated with small field sizes. The only mono-energetic proton beam facility that can provide a field up to $60 \times 60 \text{ cm}^2$ is NSRL.

Flux is also an important parameter for the selection of facilities for system-level radiation testing, in particular, for digital architectures. However, it is not always easy to find facilities that can provide the most appropriate test conditions. The problem is dual, since it may be due to (i) the pulsed time structure of the beam (that all the mentioned facilities have to various extents) and (ii) to the average flux itself, which is not always tunable over several orders of magnitude, thus not allowing to find an optimum for the system under test (SUT). For these two reasons, radiation effects strictly related to the beam configuration may occur that are not relevant for the final application. While it is still possible (in some cases) to have some play on the average flux, not much can be done for the time structure of the beam. This is because it is very rare

<figure><img src="image_6.png" /><figcaption>Fig. 3. Penetration depth of protons in aluminum in the LEO proton environment compared to that of the mixed field available at CHARM, that of spallation neutrons available at ChipIr and that from a 200-MeV mono-energetic proton beam. Only hadrons with energy above 20 MeV are considered for both space and facility environments.</figcaption></figure>
(and quite impractical) to build accelerating structures reaching 1-GeV energy with continuous beam acceleration.

In conclusion, an optimal fit for all the parameters at play to perform system-level radiation testing could not be found and, in the context of this guideline, best trade-offs have been selected in order to propose a methodology that could be implemented with what is nowadays available in terms of facilities.

If the purpose of the system-level radiation test is TID, most of the classic Co-60 sources can provide both depth-wise and surface-wide homogeneity for irradiation of a system having the aforementioned volume. The existing standard for devices [3] may be of direct application in this case.

## VI. DESIGN OF THE TEST

System-level testing can be quite challenging also in terms of test preparation. At a minimum, even when using the flight model as is, the user may be capable of observing and logging the following parameters (depending on the system input and output):

1) system-level SELs (sudden current increase over one of the power domains of the system);

2) SEFs on the main control element;

3) SETs on the output voltage/current;

4) data corruption of output data streams;

5) frequency reduction;

6) drifts of input voltage and current.

Other localized effects leading to the failure of the system (e.g., device failure due to SEB) may be identified after the test.

Increasing the radiation effect observability at a lower level (i.e., down to single critical devices) would be desirable in order to better understand radiation effects and potential remedies. However, a balance has to be kept. Overloading the system with points of measurement or code-level instrumentation may alter the original system functionality and, thus, either produce new artificial radiation effects or alter the severity or rate of occurrence of the actual system-level effects. Hence, proper testing of the radiation model of the system prior to irradiation shall be accomplished in order to exclude any malfunctioning arising from the setup itself.

Other than the intrinsic non-observability of some effects (e.g., TID-induced drifts in worst case scenarios), some effects may not be observed due to constraints imposed by how the test equipment should comply with the facility interfaces and regulations. Usually, test equipment has to be kept far from the beam, thus relying on long cables. These may lead to two undesired effects: voltage drop and signal-to-noise ratio decrease. The former resulting in an insufficient biasing of the system that may even trigger undesired setup-related effects [35]. The latter resulting in data reception corruption that is not produced within the system as a radiation effect, but rather in the cable as a parasitic setup effect.

DSEE mitigation at test-bench level should be implemented when possible, or, as an alternative, at the level of the equipment in order to avoid that unprotected effects may end

the test very early. Several mitigations can be implemented directly within the system or at the equipment level [36].

## VII. TEST LOGIC

One of the main challenges for the execution of a proper system-level radiation test is under which conditions the system has to be tested in order to provide representative information about reliability and availability, while not masking fault/failure modes due to the way the system is operated. In component-level testing, one can perform device radiation testing and data analysis based on worst case conditions (e.g., of biasing, temperature, frequency). However, when operated in a system, the devices are set to work under a specific envelope of conditions (if not just a single one that can be quite far from worst case).

While it is sometimes suggested to test under real working conditions, it is also true that systems are very rarely designed to work under a unique set of parameters or modes. For instance, when testing a satellite, this may have several different modes of operation (scientific acquisition, data downlink, telemetry and command uplink and downlink, battery recharging, etc.), which may not employ all parts of the system at the same time or may employ those parts under different loads.

At the same time, even finding a single system, worst case condition may not be so easy due to competing effects and sensitivities among devices within the system. For instance, device-level standards [1], [2] mention that the worst case for SEL would be high temperature, whereas for SEB it would be room temperature.

The situation becomes even more critical when only one SUT is available and the radiation source also provides cumulative degradation by TID/TNID. Some drivers that can help defining the best test configuration within the operating state space that would provide a representative insight for a system functional verification are:

1) the types of radiation effects that the system is expected to be prone to during the test and whose occurrence would potentially set a critical situation for the system;

2) the conditions under which the system is supposed to be operated for most of the time during its intended mission;

3) the conditions imposing the largest electrical loads on the widest set of devices;

4) performing a multipurpose test (whenever low TID/TNID is deposited in the system):

a) to exclude DSEEs;

b) running under highest data load and frequency to find upper bounds to data corruption rates.

Generally, performing a "duty-cycle" radiation testing of the satellite encompassing the various operating modes may be suitable in order to exercise the system under representative conditions whenever a clear worst case condition for the whole system cannot be found. Defining a parametric envelope for the set of variables under test can also provide a valuable option.
One critical aspect of the system-level test is to select a flux that does not lead the system into HLF with a too high rate. In addition, a moderate flux can ensure that the observed events are linear with the fluence and not the combination of accumulated events over a short time period due to the high flux of the accelerated test, which would not be representative of the low-flux conditions found in space.

The last two points are particularly critical when testing a system whose component radiation data are not known because it would be much harder to interpret whether the observed radiation effects are caused by the beam configuration or actual system faults. In order to mitigate that it is strongly advised to start the test with a low flux and then ramp it up and decide upon the observed system response. Generally, full analog systems can sustain stronger fluxes than digital systems, but it is advised not to go above $10^7$ hadrons/cm²/s. Full digital systems, on the other hand, are often plagued by flux-induced effects for fluxes higher than $10^6$ hadrons/cm²/s.

Additionally, when using the duty-cycle radiation testing scheme, it is recommended to set the flux so that, on average, at least ten consecutive duty cycles can be completed between two consecutive HLFs. For other cases, in general, it is recommended to set the flux so that the time between two HLFs is, on average, at least a 100 times the recovery time of the system.

When more than one SUT unit is available for testing (in general, for systems to be produced in hundreds or thousands of units), the worst case condition for the system can be found empirically by testing the various SUTs under different conditions. Some of the facilities previously mentioned allow performing the parallel irradiation of many systems, thanks to their broad beam. A few additional units can then be tested under the identified worst case condition to improve the level of confidence on the positive outcome of the first SUT unit.

## VIII. TEST OUTCOME AND DATA EXPLOITATION

### A. Functional Reliability

As earlier said, the test outcome on reliability is defined on a pass/fail basis. For the pass case (no PLF) against stochastic events of radiation, a cross section with $2\sigma$ level of confidence can be determined based on the statistical Poisson distribution [37] and the test fluence

$$ \sigma_{\text{PLF}} \le \frac{3.7}{\Phi_{\text{HEH}}} \text{ cm}^2/\text{system.} \quad (1) $$

The high-energy hadron (HEH) fluence $\Phi_{\text{HEH}}$ can be used as a general measurement of the flux for a hadronic environment. The general approximation behind the definition of HEH [38] is that all hadrons (in $N_p$ amount) from an energy above 20 MeV can be considered equivalent for SEE triggering, so that

$$ \Phi_{\text{HEH}} = \sum_{i=1}^{N_p} \int_{20 \text{ MeV}}^{\infty} \Phi_i(E) dE. \quad (2) $$

The main justification is that DSEEs can be triggered only by hadrons with energy above 20 MeV. Note that when combined with this fluence, the previously defined $\sigma_{\text{PLF}}$ is assumed

<table><caption>TABLE II
ALPHA FACTORS (UNITS OF cm⁻² day⁻¹) FOR THE CONSIDERED TEST FACILITIES WITH RESPECT TO THE LEO ENVIRONMENT (800 km, 98°, SOLAR MINIMUM, 100 MILS OF AL, TRAPPED PROTONS, AND GALACTIC COSMIC RAYS) FOR THE SEL-LIKE AND SEU-LIKE VOLUMES</caption><thead><tr><th></th><th>SEL SV</th><th>SEU SV</th></tr></thead><tbody><tr><td><strong>CHARM</strong></td><td>1.03 x 10<sup>7</sup></td><td>8.82 x 10<sup>6</sup></td></tr><tr><td><strong>ChipIr</strong></td><td>1.35 x 10<sup>7</sup></td><td>7.25 x 10<sup>6</sup></td></tr><tr><td><strong>200 MeV protons</strong></td><td>1.08 x 10<sup>7</sup></td><td>1.05 x 10<sup>7</sup></td></tr></tbody></table>

to be a step function starting at 20 MeV. This may generally yield higher rates than expected, although it was shown that usually these are within a factor of 1.5 for irradiations done at CHARM [39], [40].

When it comes to TID/TNID, the test outcome is pass if the system did not experience PLF up to the targeted (and potentially margined) doses. The level of confidence on the outcome can even be increased if the test is prolonged to the ultimate dose required to observe the ultimate failure of the system.

Only for failures due to unprotected SELs and unmitigated SEFIs can the user easily implement mitigation out of a fail outcome. Failure by TID/TNID and DSEEs may, on the other hand, require either part replacement or system design reiteration, which are not straightforward.

### B. Functional Availability

Cross sections for HLF and SLF can be determined similar to component-level cross sections. The actual root cause of the system-level effect may not be required to be determined (e.g., whether it was an SEFI or SEL) as long as the system is protected against permanent damage. Root causes may, on the other hand, be crucial whenever the loss of functionality rate is too high and a reduction or solution could be found by mitigation.

Other than the cross section, the typical system downtime associated with the observed interruption can be used to determine the actual availability of the system. Note that the calculated availability for the mission based on environmental fluxes, test cross sections, and downtime may be strongly impacted by the mode of operation set for the system. Hence, calculating a rate for each mode of operation may be better.

### C. Environmental Similitude for Stochastic Events

The similitude in terms of energy deposition event response among space proton-dominated environments and the proposed facility hadronic environments can allow calculation of the expected on-orbit rates whenever events are observed during the test [41], [42]. This concerns only those events that can be triggered by hadron-silicon nuclear recoils, that is, it is applicable to those devices with a low enough volume equivalent LET (LET<sub>eq,V</sub>) threshold. Not much can be said about those events that cannot be stimulated by the hadronic environment of the test and only very weak upper bounds on the worst device response to heavy ions can be applied, typically in the order of 0.01 event/device/day for mild environments and even weaker for harsher ion environments.
The on-orbit event rate can be directly obtained from the measured cross section during the test in the proposed facilities by multiplication of the latter for an appropriate factor called $\alpha$ [42] obtained from the following derivation:

$$R_{\text{test}}[\text{day}^{-1}] = \Phi_{\text{HEH}}[\text{day}^{-1}\text{cm}^{-2}] \cdot \sigma_{\text{HEH}}[\text{cm}^2] \quad (3)$$

$$\text{acc. factor} = \frac{\Phi_{\text{test}}(>\text{LET}_{\text{eq},V}^*)}{\Phi_{\text{space}}(>\text{LET}_{\text{eq},V}^*)} \quad (4)$$

$$R_{\text{space}}[\text{day}^{-1}] = \frac{R_{\text{test}}[\text{day}^{-1}]}{\text{acc. factor}} = \frac{\Phi_{\text{space}}(>\text{LET}_{\text{eq},V}^*)}{\Phi_{\text{test}}(>\text{LET}_{\text{eq},V}^*)} \cdot \Phi_{\text{HEH}} \cdot \sigma_{\text{HEH}} \quad (5)$$

$$\alpha(\text{LET}_{\text{eq},V}^*) = \Phi_{\text{space}}(>\text{LET}_{\text{eq},V}^*) \cdot \frac{\Phi_{\text{HEH}}}{\Phi_{\text{test}}(>\text{LET}_{\text{eq},V}^*)} \quad (6)$$

Note that this can be applied also to heavy-ion on-orbit rate predictions whenever the LET threshold of the device is low enough.

Fluktuierende Kaskade (FLUKA) 4.0 [43], [44] is used to perform Monte-Carlo (MC) simulations of the energy deposition event response of a LEO environment for an orbit of 800 km, 98°, solar minimum conditions, 100 mils of aluminum, and including both the trapped protons and the galactic cosmic-ray heavy ions with angular isotropic distribution. MC simulations are also used to extract the energy deposition event response for the CHARM mixed field, the ChipIr spallation neutron beam, and a mono-energetic 200-MeV proton beam. The SVs under consideration are two: a first one is representative of certain SEL structures ($20 \times 4 \ \mu\text{m}$ surface with 3 $\mu\text{m}$ thickness [45]) and the second one is representative of highly scaled SEU structures ($0.31 \times 0.31 \ \mu\text{m}$ surface with $0.31-\mu\text{m}$ thickness [46]).

Note that, unlike soft errors, the use of hadrons for the test is insufficient to screen against SELs for all those devices that may be characterized by either a low heavy-ion saturation cross section or a high LET threshold. This limitation mainly comes from the fact that hadrons are quite inefficient at producing secondary ions of sufficient $LET_{eq,V}$. For instance, about $10^4$ ions/cm$^2$ having a $LET_{eq,V} > 3$ MeV/(mg/cm$^2$) will be generated for a fluence of $10^{11}$ HEH/cm$^2$ and a volume with $3-\mu\text{m}$ thickness, that is, much lower than the typical ion fluence used for standard heavy-ion component-level testing. More ions can be produced with a higher fluence, but this has to be traded off with the increased ionizing and non-ionizing dose deposited by the primary charged hadrons. In this respect, $10^{11}$ HEH/cm$^2$ can be considered a good trade-off value considering the deposited dose of 56 Gy(Si) for a pure proton beam and the amount of secondary ions generated, also considering that little can be gained in terms of $LET_{eq,V}$ of the secondary ions themselves. The same neutron fluence will deposit less than 5 Gy(Si) while providing secondary ions in similar amount.

The choice of $3-\mu\text{m}$ thickness for the SEL SV is also representative of only a few devices, but it can be considered representative for those devices that can suffer from SELs in

<figure><img src="image_7.png" /><figcaption>Fig. 4. Event rate normalized to a device surface of 1 cm$^2$ per day for the LEO environment (800 km, 98°, solar minimum, 100 mils of Al, trapped protons, and galactic cosmic rays) compared with those from CHARM, ChipIr, and 200-MeV protons scaled by the respective alpha factors. The volume considered for the MC calculations is typical of certain SEL structures.</figcaption></figure>

hadronic environment as this size would be quite compatible with typical secondary ion ranges from hadrons' inelastic reactions. In other words, devices with thicker SVs would typically not experience events in hadronic environments, unless having an extremely low LET onset.

For these reasons, the proposed analysis has to be taken as a reliable event rate prediction tool for the space environment of concern only and solely when a significant amount of events are seen with hadron testing.

The purpose of using very different structures is to check whether a general alpha factor can be derived no matter the SV and whether this can be used to calculate on-orbit rates affecting availability even whenever the originating cause of the observed system-level fault is unknown.

One of the main assumptions is the choice of the $LET_{eq,V}^*$ at which the facility energy deposition distributions have to be scaled with respect to that of the space environment. However, $\alpha$ weakly varies for $LET_{eq,V}^*$ in the $1-10$ MeV/(mg/cm$^2$) range, regardless of the considered facility. Thus, a value of 3 MeV/(mg/cm$^2$) [42] can suitably represent both the events originating from proton indirect ionization and heavy-ion direct ionization.

The alpha factors for the two SVs and the three facilities for the mentioned LEO environment are reported in Table II. From these calculations, 200-MeV protons provide basically the same $\alpha$ regardless of the considered volumes, whereas CHARM and ChipIr are supposed to provide a lower estimate for SEU than SEL.

The respective $\alpha$ are used to calculate the plots in Figs. 4 and 5, for the SEL SV and the SEU SV, respectively. The plots show the event rate in units of SEE/day normalized to a device-sensitive surface of 1 cm$^2$. The facility rates are also divided by the acceleration factors of the test. By selecting $LET_{eq,V}^* = 3$ MeV/(mg/cm$^2$) for $\alpha$, the event rate curves tend to overlap for $LET_{eq,V} > 1$ MeV/(mg/cm$^2$) with just a small bunch of heavy-ion events at high $LET_{eq,V}$ not covered.

Fig. 6 shows the alpha factor as a function of $LET_{eq,V}$ for various facilities and SEL SVs with 3- and $10-\mu\text{m}$ thicknesses. Other than reinforcing the generality for the choice of the
<figure><img src="image_8.png" /><figcaption>Fig. 5. Event rate normalized to a device surface of 1 cm² per day for the LEO environment (800 km, 98°, solar minimum, 100 mils of Al, trapped protons, and galactic cosmic rays) compared with those from CHARM, ChipIr, and 200-MeV protons scaled by the respective alpha factors. The volume considered for the MC calculations is typical of certain highly scaled SEU structures.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Fig. 6. Alpha factor as a function of the volume equivalent LET for the SEL SV with 3 µm and 10 µm thicknesses for the various facilities.</figcaption></figure>

LET\*\_eq,v for the 3-µm thickness, the plot shows that for thicker volumes, the alpha factor diverges rather swiftly above 3 MeV/(mg/cm²). This shows that the method works well for those devices whose SEL SV is rather thin. These are usually the devices for which SELs in hadron environment can be observed. So, the validity of the method is not compromised. Devices having thicker volumes, on the other hand, usually do not display SELs in hadron environment. To this end, it is reminded that the alpha method can be considered accurate only if events are seen with hadrons, whereas not much can be concluded for 0 events.

All in all, despite the use of very different volumes and test environments, the $\alpha$ is not seen to vary much, to the point that a general value of $(1 \pm 0.3) \times 10^7 \text{ cm}^{-2} \text{ day}^{-1}$ can be assumed for the derivation of on-orbit rates from the event cross section attained at one of the proposed test facilities.

As a verification of the suitability of this method for predicting on-orbit event rates, a couple of comparisons are performed. In the first, for SEU, a few state-of-the-art devices are chosen [46]. Both the heavy-ion and high-energy proton responses of these devices are known, so that it is possible to calculate the predicted event rates through the standard Weibull fits [47] and then compare them with those

<table><caption>TABLE III

EXPECTED EVENT RATES (IN UNITS OF EVENTS/DEVICE/DAY) FOR SOME DEVICES SENSITIVE TO SEU [46] BASED ON THE WEIBULL FITS OF THE KNOWN HEAVY-ION AND HIGH-ENERGY PROTON CROSS SECTIONS AND ON THE USE OF THE $\alpha$ FACTOR FROM THE CROSS SECTIONS MEASURED IN A 200-MEV PROTON FACILITY</caption><thead><tr><th></th><th>Weibull ions and protons</th><th>α x σ<sub>proton-200MeV</sub></th></tr></thead><tbody><tr><td>ISSI</td><td>6.08</td><td>5.03</td></tr><tr><td>Cypress</td><td>16.84</td><td>13.42</td></tr><tr><td>RADSAGA</td><td>1.63</td><td>1.42</td></tr></tbody></table>

<table><caption>TABLE IV

COMPARISON OF THE EVENT RATES (IN UNITS OF EVENTS/DEVICE/DAY) FOR SOME DEVICES SENSITIVE TO SEL [42] WITH THE EXPECTED EVENT RATES FROM THE MULTIPLICATION OF THE $\alpha$ FACTOR AND THE CROSS SECTIONS MEASURED AT CHARM</caption><thead><tr><th></th><th>On-orbit rate</th><th>α x σ<sub>HEH-CHARM</sub></th></tr></thead><tbody><tr><td>IS61LV5128AL-12</td><td>1.81 x 10<sup>-1</sup></td><td>1.46 x 10<sup>-1</sup></td></tr><tr><td>K6R4008V1D</td><td>2.57 x 10<sup>-3</sup></td><td>2.66 x 10<sup>-3</sup></td></tr><tr><td>AS7C34096A</td><td>1.36 x 10<sup>-3</sup></td><td>1.73 x 10<sup>-3</sup></td></tr></tbody></table>

calculated with $\alpha = 1 \times 10^7 \text{ cm}^{-2} \text{ day}^{-1}$ and the 200-MeV proton data-point. The data are compared in Table III. For all the considered devices, the rate calculated through $\alpha$ is within ±30%, which is in agreement with the earlier specified uncertainty.

For SEL, on-orbit event rates are available [42], so that it is possible to compare the proposed prediction method with actual in-space observations. In this case, the same $\alpha = 1 \times 10^7 \text{ cm}^{-2} \text{ day}^{-1}$ is used to predict the event rate from the cross section measured at the CHARM facility. The data are compared in Table IV. The agreement with the predicted data and the in-space measured data is again within ±30%.

Note that the $\alpha$ changes with the orbit, although it is possible to calculate it for various orbits and just perform a rescaling of the value here proposed. The other limitation is that this method can work well for proton-dominated environments, which are usually those for which system-level testing is anyway best suited. Finally, the coverage is not guaranteed for SEL associated with thicker volumes (e.g., 10 µm), although it is also quite unlikely to observe any event during hadron testing for such deep SVs.

This method can be generally extended to high-level losses of functionality observed during the test to calculate their expected rate on orbit and the impact on availability.

## IX. SUMMARY AND CONCLUSION

Cost-effective radiation testing schemes adapted to the functional verification of large ensembles of devices, subsystems, and full small satellites are flourishing. A general top-down approach to functional verification through system-level radiation testing that can be employed for higher risk acceptance space missions was proposed.

Risk acceptance is the key parameter when it comes to decide whether to pursue a system-level radiation verification scheme. Several aspects of standard space qualification are intrinsically overlooked for this kind of qualification scheme, including TID WCA, ELDRS, and DD deviations from the NIEL scaling among different materials and DSEE coverage.
In addition, system-level testing of one or a few units is associated with limited level of confidence even following a pass outcome, and observability of root cause events may not always be achieved. For all these reasons, system-level radiation testing shall be seen as a tool that can cover for the functional verification of those systems lying in the gray area between “no-testing” and qualification based on standards for component-level testing. Given the long list of intrinsic limitations associated with system-level radiation testing, this methodology shall not be seen as a cheap replacement of standard qualification whenever risk acceptance would not allow so.

A taxonomy for system-level radiation effects based on their criticality was proposed. The state space of system-level radiation testing is wider than the proposed top-level functional verification given that the latter can be combined with some standard component-level qualification. However, the scope of this work was to take the very opposite end of the state space and propose guidelines on how to extract precious information about functional verification even when using the simplest radiation model possible, that is, the system “as is.” Engineering and radiation models of the system shall be consistent and should be carefully verified prior to irradiation as well as the test setup.

The proposed methodology is based on the use of deeply penetrating beams (i.e., protons, neutrons, or mixed fields) due to the intrinsic low penetration and fragmentation of ions and to the relatively large volumes considered. Currently, there are only a handful of facilities fulfilling the requirements for the irradiation of bulky systems. Other considerations related to beam characteristics such as the flux may have to be assessed when choosing the most suitable facility for the test. Sometimes, even the test infrastructure available at the facility (e.g., cabling length, test equipment shielding) may play a role on whether to pursue this kind of qualification due to radiation effects observability limitations. Other test methodologies can be built upon these general considerations to, for instance, irradiate portions of the system or by making use of heavy ions for the irradiation of single boards.

Test preparation for these kinds of tests may be as critical as the test itself. A good balance between what it is expected to be observed and in-depth observability shall be kept in order not to affect the system radiation response in other ways. Whenever the system itself is not equipped with protections from potentially destructive radiation effects, it is good practice to protect it at the level of the monitoring equipment.

Tailoring an effective test plan to fulfill all the objective of the test is also similarly important. Testing the system in a large enough set of configurations may help in identifying worst case conditions and may provide more confidence than simply testing the system in the “real” condition (which is probably just an educated guess of how the system will most likely be used). Whenever radiation data of the system and its components are not available, it is suggested to always start from a low enough flux and ramp up only when flux-dependent events cannot be observed. In addition, the frequency of HLFs shall be much smaller than the standard duty-cycle execution time of the system.

Data exploitation in terms of reliability comes with a limited level of confidence whenever a single unit is tested. More confidence can be built on availability provided a sufficient amount of events is observed with hadrons and for those mission environments dominated by protons. The environmental similitude among the test facility energy deposition environments and those of certain space orbits (both in terms of proton and heavy-ion energy depositions) can, if events are seen during the test, allow the calculation of expected on-orbit rates relying on the use of the alpha factor method. This was shown to return mission rates compatible with actual on-orbit measurements and the classic Weibull prediction. While the alpha factor varies with the orbit, it was shown to vary just slightly among the different hadronic beams and SVs considered, meaning that the alpha factor could be easily rescaled just based on the orbit. Nonetheless, it shall be borne in mind that the hadron test may be blind to certain potentially destructive events triggered by heavy ions and that the proposed rate calculation method can be used to predict only and solely events that could be observed during the test with hadrons.

## ACKNOWLEDGMENT

The authors recognize the valuable feedback provided by Renaud Mangeret from Airbus, Françoise Bezerra from CNES, and Jonathan Pellish and Anthony Sanders from NASA during the write-up of the guideline document related to this article.

## REFERENCES

[1] Single Event Effects Test Method and Guidelines, Eur. Space Compon. Coordination, ESCC 25100, ESA, Paris, France, Oct. 2014.

[2] Single-Event Burnout and Single-Event Gate Rupture, document MIL-STD-750-1 Method 1080.1, United States Dept. Defense, Nov. 2006.

[3] Total Dose Steady-State Irradiation Test Method, Eur. Space Compon. Coordination, ESCC 22900, ESA, Paris, France, Oct. 2010.

[4] P. M. O'Neill, G. D. Badhwar, and W. X. Culpepper, “Risk assessment for heavy ions of parts tested with protons,” *IEEE Trans. Nucl. Sci.*, vol. 44, no. 6, pp. 2311–2314, Dec. 1997.

[5] K. A. LaBel et al., “Radiation evaluation method of commercial off-the-shelf (COTS) electronic printed circuit boards (PCBs),” in *Proc. 5th Eur. Conf. Radiat. Effects Compon. Syst. (RADECS)*, Fontevraud, France, Sep. 1999, pp. 528–534.

[6] S. M. Guertin. Board Level Proton Testing Book of Knowledge for NASA Electronic Parts and Packaging Program. Accessed: Oct. 2018. [Online]. Available: https://trs.jpl.nasa.gov/handle/2014/45964

[7] S. M. Guertin, “Lessons and recommendations for board-level testing with proton,” in *Proc. Small Satell. Conf.*, Logan, UT, USA, 2018, pp. 1–7. Accessed: Jun. 2019. [Online]. Available: https://digitalcommons.usu.edu/smallsat/2018/all2018/446/

[8] R. Ladbury, J.-M. Lauenstein, and K. P. Hayes, “Use of proton SEE data as a proxy for bounding heavy-ion SEE susceptibility,” *IEEE Trans. Nucl. Sci.*, vol. 62, no. 6, pp. 2505–2510, Dec. 2015.

[9] R. L. Ladbury and J.-M. Lauenstein, “Evaluating constraints on heavy-ion SEE susceptibility imposed by proton SEE testing and other mixed environments,” *IEEE Trans. Nucl. Sci.*, vol. 64, no. 1, pp. 301–308, Jan. 2017.

[10] R. Mangeret, “Radiation hardness assurance: How well assured do we need to be?” in *Proc. NSREC Short Course, Part II*, Waikoloa, HI, USA, Jul. 2018, pp. 1–64.

[11] S. Uznanski et al., “Qualification of electronics components for a radiation environment: When standards do not exist,” in *Proc. RADECS Short Course, Part 4B*, Geneva, Switzerland, Oct. 2017, pp. 1–55.

[12] F. Bezerra et al., “Evaluation of an alternative low cost approach for SEE assessment of a SoC,” in *Proc. 17th Eur. Conf. Radiat. Effects Compon. Syst. (RADECS)*, Geneva, Switzerland, Oct. 2017, pp. 418–422.
[13] A. Coronetti et al., "Mixed-field radiation qualification of a COTS space on-board computer along with its CMOS camera payload," in *Proc. RADECS Conf.*, 2019.

[14] J. Budroweit, S. Mueller, M. Jaksch, R. G. Alfa, A. Coronetti, and A. Koelpin, "In-situ testing of a multi-band software-defined radio platform in a mixed-field irradiation environment," *Aerospace*, vol. 6, no. 10, Sep. 2019, Art. no. 106.

[15] R. Secondo et al., "System level radiation characterization of a 1U CubeSat based on CERN radiation monitoring technology," *IEEE Trans. Nucl. Sci.*, vol. 65, no. 8, pp. 1694-1699, Aug. 2018.

[16] ESA COTS Initiative, WG2-3 Synthesis, ESA-ESTEC, Noordwijk, The Netherlands, Nov. 2019.

[17] *Electropedia*, IEC. Accessed: Jan. 2020. [Online]. Available: http://www.electropedia.org

[18] S. Uznanski, B. Todd, A. Dinius, Q. King, and M. Brugger, "Radiation hardness assurance methodology of radiation tolerant power converter controls for large hadron collider," *IEEE Trans. Nucl. Sci.*, vol. 61, no. 6, pp. 3694-3700, Dec. 2014.

[19] Guidelines for Verification Strategies to Minimize Risk Based on Mission Environment, Application and Lifetime (MEAL). NASA/TM-2018-220074. Accessed: Jan. 2021. [Online]. Available: https://ntrs.nasa.gov/citations/20180007514

[20] C. Weulersee et al., "Preliminary guidelines and predictions for 14-MeV neutron SEE testing," *IEEE Trans. Nucl. Sci.*, vol. 64, no. 8, pp. 2268-2275, Aug. 2017.

[21] ECSS-Q-ST-30-11C: Space Product Assurance—Derating EEE Components, ESA-ESTEC, Noordwijk, The Netherlands, 2011.

[22] International Electrotechnical Commission, Process Management for Avionics—Atmospheric Radiation Effects—Part 4: Design of Hugh Voltage Aircraft Electronics Managing Potential Single Event Effects, Standard IEC 62396-4, Sep. 2013.

[23] M. Campola and J. Pellish, "Radiation hardness assurance: Evolving for new space," in *Proc. RADECS Short Course, Part V*, Montpellier, France, Sep. 2019, pp. 1-35.

[24] M. R. Shaneyfelt et al., "Thermal-stress effects and enhanced low dose rate sensitivity in linear bipolar ICs," *IEEE Trans. Nucl. Sci.*, vol. 47, no. 6, pp. 2539-2545, Dec. 2000.

[25] R. L. Pease, "2008 update to the ELDRS bipolar linear circuit data compendium," in *Proc. Eur. Conf. Radiat. Effects Compon. Syst.*, Jyväskylä, Finland, Sep. 2008, pp. 75-78.

[26] R. Ferraro et al., "COTS optocoupler radiation qualification process for LHC applications based on mixed-field irradiations," *IEEE Trans. Nucl. Sci.*, vol. 67, no. 7, pp. 1395-1403, Jul. 2020.

[27] F. Irom. *Guideline for Ground Radiation Testing of Microprocessors in the Space Radiation Environment*. Accessed: Mar. 2019. [Online]. Available: https://trs.jpl.nasa.gov/handle/2014/40790

[28] SEECA, Single Event Effect Criticality Analysis. NASA. Accessed: Jan. 2021. [Online]. Available: https://radhome.gsfc.nasa.gov/radhome/papers/seeca2.htm

[29] NASA Space Radiation Laboratory, Brookhaven National Laboratory. Accessed: Nov. 2019. [Online]. Available: https://www.bnl.gov/nsrl/userguide/

[30] J. F. Ziegler and J. P. Biersack, *Stopping and Range of Ions in Matter*. Accessed: Aug. 2018. [Online]. Available: http://www.srim.org

[31] A. de Bibikoff and P. Lamberbourg, "Method for system-level testing of COTS electronic board under high-energy heavy ions," *IEEE Trans. Nucl. Sci.*, vol. 67, no. 10, pp. 2179-2187, Oct. 2020.

[32] C. Cazzaniga and C. D. Frost, "Progress of the scientific commissioning of a fast neutron beamline for chip irradiation," *J. Phys., Conf. Ser.*, vol. 1021, May 2018, Art. no. 012037.

[33] J. Mekki, M. Brugger, R. G. Alia, A. Thornton, N. C. D. S. Mota, and S. Danzeca, "CHARM: A mixed field facility at CERN for radiation tests in ground, atmospheric, space and accelerator representative environments," *IEEE Trans. Nucl. Sci.*, vol. 63, no. 4, pp. 2106-2114, Aug. 2016.

[34] CREME, Vanderbilt University. Accessed: Jul. 2020. [Online]. Available: https://creme.isde.vanderbilt.edu

[35] T. Rajkowski et al., "Analysis of SET propagation in a system in package point of load converter," *IEEE Trans. Nucl. Sci.*, vol. 67, no. 7, pp. 1494-1502, Jul. 2020.

[36] H. Quinn, "Challenges in testing complex systems," *IEEE Trans. Nucl. Sci.*, vol. 61, no. 2, pp. 766-786, Apr. 2014.

[37] G. M. Swift, "SEE testing lessons from Dickens, Scounting and Oz," in *Proc. SEE Symp.*, May 2006.

[38] K. Roed et al., "Method for measuring mixed field radiation levels relevant for SEEs at the LHC," *IEEE Trans. Nucl. Sci.*, vol. 59, no. 4, pp. 1040-1047, Aug. 2012.

[39] A. Coronetti et al., "The pion single-event effect resonance and its impact in an accelerator environment," *IEEE Trans. Nucl. Sci.*, vol. 67, no. 7, pp. 1606-1613, Jul. 2020.

[40] N. Kerboub et al., "Comparison between in-flight SEL measurement and ground estimation using different facilities," *IEEE Trans. Nucl. Sci.*, vol. 66, no. 7, pp. 1541-1547, Jul. 2019.

[41] C. C. Foster, P. M. O'Neill, and C. K. Kouba, "Risk assessment based on upset rates from high energy proton tests and Monte Carlo simulations," *IEEE Trans. Nucl. Sci.*, vol. 55, no. 6, pp. 2962-2969, Dec. 2008.

[42] R. G. Alia et al., "Simplified SEE sensitivity screening for COTS components in space," *IEEE Trans. Nucl. Sci.*, vol. 64, no. 2, pp. 882-890, Feb. 2017.

[43] G. Battistoni et al., "Overview of the FLUKA code," *Ann. Nucl. Energy*, vol. 82, pp. 10-18, Aug. 2015.

[44] T. T. Böhlen et al., "The FLUKA code: Developments and challenges for high energy and medical applications," *Nucl. Data Sheets*, vol. 120, pp. 211-214, Jun. 2014.

[45] R. G. Alia et al., "SEL cross section energy dependence impact on the high energy accelerator failure rate," *IEEE Trans. Nucl. Sci.*, vol. 61, no. 6, pp. 2936-2944, Dec. 2014.

[46] A. Coronetti et al., "Assessment of proton direct ionization for the radiation hardness assurance of deep sub-micron SRAMs used in space applications," *IEEE Trans. Nucl. Sci.*, to be published.

[47] E. L. Petersen, J. C. Pickel, J. H. Adams, and E. C. Smith, "Rate prediciton for single event effects," *IEEE Trans. Nucl. Sci.*, vol. 39, no. 6, pp. 1577-1599, Dec. 1992.