of the long and short wavelength channels. SPRINTS combines the pre-eruptive forecast probabilities of MAG4 with SPRINTS post-eruptive forecast probabilities to provide continuously-updated and coupled forecast probabilities of SPEs as new phenomena and events are observed. It is designed as an extensible forecast application, where new datasets and new models can be integrated into the system's database and forecast modeling processes. The model currently runs automatically and in real-time on an Amazon Web Services GovCloud environment.

For its SPE primary forecast model, SPRINTS uses MAG4's X-ray flare forecast probabilities and feeds them into a user-defined ML model. The currently running ML model is the Multi-Layer Perceptron (MLP) model but has options to run other models including random forecasts and SVM. The MLP model is trained to predict SPEs at user-defined proton energy channels (e.g., 1, 5, 10, 30, 50, 100 MeV), fluxes, and time resolution (e.g., 12-h) out to 96 h.

To train the SPRINTS models, the data first needed to be labeled. Flare-only events were all labeled as a “zero”, while the flare events that led to an SPE were labeled as a “one”. The labeling was done per the desired temporal resolution, i.e., if the SPE occurred in the first 24 h after the flare onset, the 24-h bin was labeled as a one, while all other temporal resolutions were labeled as a zero.

As expected, the labeled data was unbalanced and heavily weighted towards flare-only events. For example, the maximum number of flares with SPEs was 152 for the proton events reaching 10 pfu at 50 MeV in a 96-h bin, whereas the number of flare-only events was 19,959. Because the problem of detecting SPEs suffers from the issue of unbalanced data, the selection of the training data is important to ensure the accuracy of the models.

To select the training data, the flares were first subsampled down to only include those flares with an SPE fluence above the minimum threshold for SPEs. This left 10,084 flare-only events. It was also empirically found that a 1 to 3 ratio of SPE to flare events resulted in the best models. For training the models, the SPEs were up sampled to a 1 to 3 ratio. The models were then trained through cross-validation using 4-folds. Results are reported for all flare and SPEs within the catalog.

Prior to an eruption of a flare, SPRINTS takes the MAG4 flare probabilities and feeds them into the MLP model where it then provides the probabilities of the user defined SPEs. As a flare is detected, the forecast transitions from the pre-eruptive forecast mode to the post- (or ongoing) eruptive mode. The flare metadata parameters are captured on-the-fly throughout the course of the flare and feed into the model every 5 min. Once the flare has ended, those parameters are “locked” in and the post-eruptive forecast is final. SPRINTS provides the pre-eruptive (MAG4), post-eruptive, and coupled forecasts through its REST API.

**Inputs:** SPRINTS ingests everything MAG4 needs and GOES X-ray and proton data. An automatic flare event

detection script Aschwanden and Freeland (2012) identifies flare start, peak, and end times. SPE predictions are made by the MLP model based on the metadata from the MAG4 predicted or observed X-ray flare events (peak flux, fluence, peak ratio) and eruptive location provided by the Latest Events Pipeline from Lockheed Martin Solar and Astrophysics Laboratory (LMSAL)) or, optionally, NOAA SWPC.

**Outputs:**

* Pre-eruptive (updated MAG4)

- M- and X-Class Flares (24 and 48 h)

- X-class Flares (24 and 48 h)

- X-ray fluences (24 and 48 h)

- X-ray peak ratio (24 and 48 h)

- CMEs (24 and 48 h)

- Fast CMEs (24 and 48 h)

* Post-eruptive and coupled

- SPEs (96 h at 12-h temporal resolution)

* Currently deployed in real-time for: 10 MeV @ 10 pfu, 10 @ 40, 30 @ 10, 50 @ 10, 100 @ 1

* Configurable to any desired GOES energy channel and flux threshold

**Free parameters:** None.

**Limitations and caveats:** Fig. 16 shows the SPRINTS primary forecast dashboard. X-ray, proton, and solar wind observations are displayed for the past 8 days with 4-day forecast panels directly to the right. For the observed SPE for this time-period you can see what the forecast probabilities were for the five SPE criteria at a 12-h resolution out to 96 h from the peak of the flare.

We note that the red forecast bar in the first 12-h bin is for the 30 MeV channel at 10 pfu and is a higher probability than the 10 MeV channel at 10 pfu. This is because the forecast capability is comprised of multiple MLP models – one for each temporal bin and energy/flux. Therefore, this results in 20 MLP independent models.

As with many ML models the unbalanced nature of the number of flares to SPEs leads to problems during the training of the model. The SPRINTS team is currently developing a time-series ML model to address this issue.

For the forecast example shown for the May 2012 event, the flare occurred close to the western limb. This was well outside of MAG4's optimal active region observation window of 45 heliocentric degrees. This highlights the benefit of combining the MAG4 pre-eruptive capability with a post-eruptive capability within SPRINTS.

**Access to model output or forecasts:** Access to model outputs and supporting dashboards can be obtained by becoming IP whitelisted to NextGen's REST API. Email aengell@nextgenfed.com to be whitelisted. NextGen is currently deploying the real-time dashboard on a public website and will be available in 2022. Additionally, SPRINTS will be providing forecasts on the SEP Scoreboard.
<figure><img src="image_17.png" /><figcaption>Fig. 16. SPRINTS SPE post-eruptive probabilistic forecasts displayed on top of proton observations. MAG4 flare X and $M^+X$ forecast probabilities are displayed.</figcaption></figure>

**Model validation:** Model validation has been performed on the post-eruptive modeling capabilities of SPRINTS. Fig. 17 shows the POD, FAR, and HSS results for 24 h temporal bins. SPRINTS is under consideration for inclusion in a comparative evaluation of several SEP forecasting models being performed by the Air Force Institute of Technology. SPRINTS model validation is a straight-forward process that is baked into model development and deployment cycle.

SPRINTS collaborative JupyterHub environment provides executable code to evaluators and supportive dashboards such as the one shown in Fig. 16. Parallel coordinates, 4D plots, and scatterplot matrices are among the plotting tools to support SEP event occurrence and ML modeling understanding.

### 3.34. STAT - Solar Particle Event (SPE) Threat Assessment Tool

**Model developers:** Jon Linker, Ron Caplan, Cooper Downs, Tibor Török, Roberto Lionello, Viacheslav Titov, Erika Palmerio (Predictive Science Inc.).

Nathan Schwadron, Matthew Gorby, Matthew Young (University of New Hampshire).

**Model description:** The SPE Threat Assessment Tool (STAT) (Linker et al., 2019; Young et al., 2021) is developed by Predictive Science Inc (PSI) and University of New Hampshire. STAT combines the Energetic Particle Radiation Environment Module (EPREM, a component of EMMREM Schwadron et al. (2010)) with the Magnetohydrodynamic Algorithm outside a Sphere (MAS, a component of CORona-HELiosphere (COR-HEL), Riley et al. (2012)). STAT utilizes precomputed MAS simulations of CMEs to simulate SEP events

and provide diagnostics that can be compared with observations.

MAS is an MHD model that simulates the structure of the plasma in the corona and inner heliosphere. It handles the background corona conditions, CME formation, and CME eruption. MAS integrates the standard viscous and resistive one-fluid MHD equations in 3D spherical coordinates. It solves the thermodynamic MHD equations which extend the standard equations by including terms for thermal conduction, radiative losses, and coronal heating (Lionello et al., 2009; Mikić et al., 2018). In this approach, the plasma density and temperature at the lower boundary are chosen to be uniform and roughly consistent with the upper chromosphere, and the coronal heating specification determines the properties (density, temperature, velocity) of the simulated coronal and solar wind plasma. The coronal heating can either be empirical (Lionello et al., 2009), or based on a Wave-Turbulence Driven model (Downs et al., Dec. 2016).

EPREM (see Section 3.7) is a 3D kinetic model that simulates particle transport anywhere in the heliosphere. It uses a Lagrangian grid scheme such that the nodes where information is stored move with the plasma. EPREM solves the FTE which describes how the distribution of particles changes with time and includes terms for convection, parallel diffusion, adiabatic focusing, adiabatic cooling, and pitch-angle scattering. There is a separate module within EPREM that solves for perpendicular diffusion and particle drift.

STAT starts with a global magnetic map of the sun's radial magnetic field derived from magnetogram observations. The majority of grid cells are concentrated within the AR of interest, which may incorporate a higher resolution vector or LOS magnetogram. The areas around the
<figure><img src="image_18.png" /><figcaption>Fig. 16. SPRINTS SPE post-eruptive probabilistic forecasts displayed on top of proton observations. MAG4 flare X and M + X forecast probabilities are displayed.</figcaption></figure>

**Model validation:** Model validation has been performed on the post-eruptive modeling capabilities of SPRINTS. Fig. 17 shows the POD, FAR, and HSS results for 24 h temporal bins. SPRINTS is under consideration for inclusion in a comparative evaluation of several SEP forecasting models being performed by the Air Force Institute of Technology. SPRINTS model validation is a straight-forward process that is baked into model development and deployment cycle.

SPRINTS collaborative JupyterHub environment provides executable code to evaluators and supportive dashboards such as the one shown in Fig. 16. Parallel coordinates, 4D plots, and scatterplot matrices are among the plotting tools to support SEP event occurrence and ML modeling understanding.

### 3.34. STAT - Solar Particle Event (SPE) Threat Assessment Tool

**Model developers:** Jon Linker, Ron Caplan, Cooper Downs, Tibor Török, Roberto Lionello, Viacheslav Titov, Erika Palmerio (Predictive Science Inc.).

Nathan Schwadron, Matthew Gorby, Matthew Young (University of New Hampshire).

**Model description:** The SPE Threat Assessment Tool (STAT) (Linker et al., 2019; Young et al., 2021) is developed by Predictive Science Inc (PSI) and University of New Hampshire. STAT combines the Energetic Particle Radiation Environment Module (EPREM, a component of EMMREM Schwadron et al. (2010)) with the Magnetohydrodynamic Algorithm outside a Sphere (MAS, a component of CORona-HELiosphere (COR-HEL), Riley et al. (2012)). STAT utilizes precomputed MAS simulations of CMEs to simulate SEP events

and provide diagnostics that can be compared with observations.

MAS is an MHD model that simulates the structure of the plasma in the corona and inner heliosphere. It handles the background corona conditions, CME formation, and CME eruption. MAS integrates the standard viscous and resistive one-fluid MHD equations in 3D spherical coordinates. It solves the thermodynamic MHD equations which extend the standard equations by including terms for thermal conduction, radiative losses, and coronal heating (Lionello et al., 2009; Mikić et al., 2018). In this approach, the plasma density and temperature at the lower boundary are chosen to be uniform and roughly consistent with the upper chromosphere, and the coronal heating specification determines the properties (density, temperature, velocity) of the simulated coronal and solar wind plasma. The coronal heating can either be empirical (Lionello et al., 2009), or based on a Wave-Turbulence Driven model (Downs et al., Dec. 2016).

EPREM (see Section 3.7) is a 3D kinetic model that simulates particle transport anywhere in the heliosphere. It uses a Lagrangian grid scheme such that the nodes where information is stored move with the plasma. EPREM solves the FTE which describes how the distribution of particles changes with time and includes terms for convection, parallel diffusion, adiabatic focusing, adiabatic cooling, and pitch-angle scattering. There is a separate module within EPREM that solves for perpendicular diffusion and particle drift.

STAT starts with a global magnetic map of the sun's radial magnetic field derived from magnetogram observations. The majority of grid cells are concentrated within the AR of interest, which may incorporate a higher resolution vector or LOS magnetogram. The areas around the
<table><caption>Fig. 17. Table of SPRINTS validation results.</caption><thead><tr><th>AFRL req.</th><th>HSS</th><th>POD</th><th>FAR</th></tr></thead><tbody><tr><td>10@10</td><td>0.58</td><td>0.56</td><td>0.34</td></tr><tr><td>10@40</td><td>0.73</td><td>0.59</td><td>0.00</td></tr><tr><td>30@10</td><td>0.66</td><td>0.56</td><td>0.17</td></tr><tr><td>50@10</td><td>0.80</td><td>0.67</td><td>0.00</td></tr><tr><td>100@1</td><td>0.89</td><td>0.80</td><td>0.00</td></tr></tbody></table>

poles, where no measurements exist or are not reliable, are fitted in the magnetic map using an extrapolation technique (Linker et al., 2013) or using the polar filling provided by observatories (e.g., Sun et al., 2011). The data is interpolated onto the non-uniform numerical grid of the lower boundary using an integral (flux) preserving interpolation scheme, and smoothed to the grid resolution (Linker et al., 2017).

To create the coronal and solar wind background, the thermodynamic MHD equations are solved on a non-uniform spherical mesh that ranges from 1 solar radii to 20–30 solar radii, until the solar wind has fully opened up the field associated with coronal holes, resulting in a steady-state MHD solution. The coronal heating specification is chosen to give a reasonable match to the observed EUV and SXR emission.

To initiate the CME, a stable magnetic flux rope is constructed along the AR’s Polarity Inversion Line (PIL) and inserted into the global thermodynamic MHD solution in a way that the original magnetogram is preserved. In practice, the configuration is first relaxed toward a force-free state using a zero-beta MHD solution prior to insertion. Török et al. (2018) described the use of multiple, modified Titov-Démoulin (TDm) flux ropes (Titov et al., 2014) to model the July 14, 2000 flare/CME. In that case, the eruption was triggered by imposing small photospheric flows that canceled flux at the PIL, resulting in a slow rise and successive detachment of the flux rope until it became unstable and erupted. More recent examples use the Regularized Biot-Savart Laws (RBSL) flux rope model (Titov et al., 2018), that preserves the magnetogram flux by construction. The eruption can also be initiated by setting the axial flux of the rope slightly above the stable value, leading to a similar eruptive sequence.

Particle transport and particle energization is handled within STAT by EPREM. To begin, a seed population using a differential flux that scales as a power law in energy with an exponential roll-over is assumed. This injected spectrum is converted into a distribution function and acts

as the source term in the FTE. EPREM’s grid nodes spawn on a rotating inner boundary at the surface of the sun. The nodes are then convected out with the solar wind. As the sun rotates, the nodes naturally make a Parker spiral configuration. The structure of the CME and subsequent effects on the IMF push the nodes around according to the MHD equations solved within MAS.

At each time-step within EPREM, the model solves the FTE which describes how the distribution of particles changes with time. The FTE includes terms for convection, parallel diffusion, adiabatic focusing, adiabatic cooling and heating, and pitch-angle scattering. There is a separate module within EPREM that solves for perpendicular diffusion and particle drift; these terms were turned off for the events that have been delivered to CCMC. Each node contains information about the proton flux at the location of the node and its time-history. The user can select a node near Earth or other locations of interest to access the proton flux time-profile.

The STAT model aims to reproduce the complete set of physics governing the coronal eruption and the acceleration and transport of energetic particles. This model is well-suited to testing and improving our understanding of the critical physics associated with the production and distribution of solar energetic particles.

**Inputs:**

* Global magnetic map, either an observatory synoptic map or from a flux transport model

* Coronal heating specification, parameters chosen to specify flux rope and CME eruption

* Seed population, parallel mean free path, perpendicular diffusion

**Outputs:**

* Differential fluxes on a fine energy grid at any node

* >10, >50, and >100 MeV integral proton flux time profile at any node.
<table><caption>Fig. 17. Table of SPRINTS validation results.</caption><thead><tr><th>AFRL req.</th><th>HSS</th><th>POD</th><th>FAR</th></tr></thead><tbody><tr><td>10@10</td><td>0.58</td><td>0.56</td><td>0.34</td></tr><tr><td>10@40</td><td>0.73</td><td>0.59</td><td>0.00</td></tr><tr><td>30@10</td><td>0.66</td><td>0.56</td><td>0.17</td></tr><tr><td>50@10</td><td>0.80</td><td>0.67</td><td>0.00</td></tr><tr><td>100@1</td><td>0.89</td><td>0.80</td><td>0.00</td></tr></tbody></table>

poles, where no measurements exist or are not reliable, are fitted in the magnetic map using an extrapolation technique (Linker et al., 2013) or using the polar filling provided by observatories (e.g., Sun et al., 2011). The data is interpolated onto the non-uniform numerical grid of the lower boundary using an integral (flux) preserving interpolation scheme, and smoothed to the grid resolution (Linker et al., 2017).

To create the coronal and solar wind background, the thermodynamic MHD equations are solved on a non-uniform spherical mesh that ranges from 1 solar radii to 20–30 solar radii, until the solar wind has fully opened up the field associated with coronal holes, resulting in a steady-state MHD solution. The coronal heating specification is chosen to give a reasonable match to the observed EUV and SXR emission.

To initiate the CME, a stable magnetic flux rope is constructed along the AR’s Polarity Inversion Line (PIL) and inserted into the global thermodynamic MHD solution in a way that the original magnetogram is preserved. In practice, the configuration is first relaxed toward a force-free state using a zero-beta MHD solution prior to insertion. Török et al. (2018) described the use of multiple, modified Titov-Démoulin (TDm) flux ropes (Titov et al., 2014) to model the July 14, 2000 flare/CME. In that case, the eruption was triggered by imposing small photospheric flows that canceled flux at the PIL, resulting in a slow rise and successive detachment of the flux rope until it became unstable and erupted. More recent examples use the Regularized Biot-Savart Laws (RBSL) flux rope model (Titov et al., 2018), that preserves the magnetogram flux by construction. The eruption can also be initiated by setting the axial flux of the rope slightly above the stable value, leading to a similar eruptive sequence.

Particle transport and particle energization is handled within STAT by EPREM. To begin, a seed population using a differential flux that scales as a power law in energy with an exponential roll-over is assumed. This injected spectrum is converted into a distribution function and acts

as the source term in the FTE. EPREM’s grid nodes spawn on a rotating inner boundary at the surface of the sun. The nodes are then convected out with the solar wind. As the sun rotates, the nodes naturally make a Parker spiral configuration. The structure of the CME and subsequent effects on the IMF push the nodes around according to the MHD equations solved within MAS.

At each time-step within EPREM, the model solves the FTE which describes how the distribution of particles changes with time. The FTE includes terms for convection, parallel diffusion, adiabatic focusing, adiabatic cooling and heating, and pitch-angle scattering. There is a separate module within EPREM that solves for perpendicular diffusion and particle drift; these terms were turned off for the events that have been delivered to CCMC. Each node contains information about the proton flux at the location of the node and its time-history. The user can select a node near Earth or other locations of interest to access the proton flux time-profile.

The STAT model aims to reproduce the complete set of physics governing the coronal eruption and the acceleration and transport of energetic particles. This model is well-suited to testing and improving our understanding of the critical physics associated with the production and distribution of solar energetic particles.

**Inputs:**

* Global magnetic map, either an observatory synoptic map or from a flux transport model

* Coronal heating specification, parameters chosen to specify flux rope and CME eruption

* Seed population, parallel mean free path, perpendicular diffusion

**Outputs:**

* Differential fluxes on a fine energy grid at any node

* >10, >50, and >100 MeV integral proton flux time profile at any node.
March 7, 2012 Particle Simulation with Focused Transport

<figure><img src="image_18.png" /></figure>

Locations Near Earth

<caption>Fig. 18. STAT simulation of the March 7, 2012 CME/SEP event, showing showing particles widely distributed in longitude. (a) Integrated proton flux >10 MeV as a function of longitude and sine(latitude) at 1 AU. This is a standard output visualization from STAT. The circles indicate the location of EPREM nodes where the calculation is performed. The locations of the two STEREO spacecraft and Earth are shown. (b) Comparison of >10 MeV integrated proton flux with GOES.</caption>

* Animated map in longitude and sin(latitude) of >10, >50, or >100 MeV integral proton flux using all nodes at the coronal boundary and at 1 AU.

* Animated map in radius and longitude (slice of the ecliptic plane) of >10, >50, or >100 MeV integral proton flux using all nodes from the sun out to the coronal boundary or 1 AU.

Fig. 18 shows model output for the March 7, 2012 CME, a strong event that produced SEPs that were measured at multiple locations in the heliosphere (Kouloumvakos et al., 2016). Fig. 18(a) shows the >10 MeV integrated proton flux as a function of longitude and latitude at 1 AU. Fig. 18(b) shows a comparison to GOES data for the event.

**Free parameters (MHD):** coronal heating specification, flux rope specification.

**Free parameters (FTE):** Seed population spectral shape and normalization, parallel mean free path, perpendicular diffusion.

**Limitations and caveats:** In terms of forecasting, STAT is a computationally intensive physics model that at present does not run in real time. To use STAT in an operational environment, it may be feasible to follow the approach of other physics-based models, such as SPARX (Section 3.31), and create a database of pre-run historical SEP events that can be accessed to generate a forecast.

At the time of writing, CME propagation within STAT is limited to the boundary of the coronal domain at 20–30 R⊙. Once the CME has left this domain, particle acceleration from CME propagation in the heliosphere is not modeled, restricting the simulation to only the first few hours of the SEP event (Young et al., 2021).

**Access to model output or forecasts:** STAT may be run on demand using CCMC's runs-on-request system. Some pre-run model simulations produced by PSI are available at https://ccmc.gsfc.nasa.gov/ISEP/.

**Model validation:** Young et al. (2021) presented a simulation of the 14 July 2000 Bastille Day event compared to measured GOES-08 fluxes. The comparison included both nominal GOES-08 data provided by NOAA and corrected fluxes produced using recalibrated GOES-08 energy bins derived by Sandberg et al. (2014).

### 3.35. UMASEP - University of Malaga Solar Energetic Particles

**Model developers:** Marlon Nunez (University of Malaga).

**Model description:** The University of Malaga Solar Energetic Particles (UMASEP) models are an empirical suite of tools comprised of multiple independent modules predicting different particle energy ranges: UMASEP-10 (Núñez, 2011) predicts >10 MeV SEP events; UMASEP-
March 7, 2012 Particle Simulation with Focused Transport

<figure><img src="image_19.png" /></figure>

Locations Near Earth

<caption>Fig. 18. STAT simulation of the March 7, 2012 CME/SEP event, showing showing particles widely distributed in longitude. (a) Integrated proton flux >10 MeV as a function of longitude and sine(latitude) at 1 AU. This is a standard output visualization from STAT. The circles indicate the location of EPREM nodes where the calculation is performed. The locations of the two STEREO spacecraft and Earth are shown. (b) Comparison of >10 MeV integrated proton flux with GOES.</caption>

* Animated map in longitude and sin(latitude) of >10, >50, or >100 MeV integral proton flux using all nodes at the coronal boundary and at 1 AU.

* Animated map in radius and longitude (slice of the ecliptic plane) of >10, >50, or >100 MeV integral proton flux using all nodes from the sun out to the coronal boundary or 1 AU.

Fig. 18 shows model output for the March 7, 2012 CME, a strong event that produced SEPs that were measured at multiple locations in the heliosphere (Kouloumvakos et al., 2016). Fig. 18(a) shows the >10 MeV integrated proton flux as a function of longitude and latitude at 1 AU. Fig. 18(b) shows a comparison to GOES data for the event.

**Free parameters (MHD):** coronal heating specification, flux rope specification.

**Free parameters (FTE):** Seed population spectral shape and normalization, parallel mean free path, perpendicular diffusion.

**Limitations and caveats:** In terms of forecasting, STAT is a computationally intensive physics model that at present does not run in real time. To use STAT in an operational environment, it may be feasible to follow the approach of other physics-based models, such as SPARX (Section 3.31), and create a database of pre-run historical SEP events that can be accessed to generate a forecast.

At the time of writing, CME propagation within STAT is limited to the boundary of the coronal domain at 20–30 R⊙. Once the CME has left this domain, particle acceleration from CME propagation in the heliosphere is not modeled, restricting the simulation to only the first few hours of the SEP event (Young et al., 2021).

**Access to model output or forecasts:** STAT may be run on demand using CCMC's runs-on-request system. Some pre-run model simulations produced by PSI are available at https://ccmc.gsfc.nasa.gov/ISEP/.

**Model validation:** Young et al. (2021) presented a simulation of the 14 July 2000 Bastille Day event compared to measured GOES-08 fluxes. The comparison included both nominal GOES-08 data provided by NOAA and corrected fluxes produced using recalibrated GOES-08 energy bins derived by Sandberg et al. (2014).

### 3.35. UMASEP - University of Malaga Solar Energetic Particles

**Model developers:** Marlon Nunez (University of Malaga).

**Model description:** The University of Malaga Solar Energetic Particles (UMASEP) models are an empirical suite of tools comprised of multiple independent modules predicting different particle energy ranges: UMASEP-10 (Núñez, 2011) predicts >10 MeV SEP events; UMASEP-
100 (Núñez, 2015) predicts >100 MeV events; and UMASEP-500 (Núñez et al., 2017) predicts the occurrence of a GLE. UMASEP-500 was developed in collaboration with the European Unions's HESPERIA project, which also includes the RELEASE code (Section 3.21). Two additional modules, UMASEP-30 and UMASEP-50, have recently been developed to forecast >30 and >50 MeV events. All model versions use similar underlying algorithms; however, the implementation differs.

The preliminary forecast functionality of UMASEP-10 was originally divided into the Well-Connected (WC) and Poorly-Connected (PC) modules. The WC module assesses the relationship between the time series GOES SXR (X-ray emissions) and an increase in GOES differential proton flux to determine if particles have escaped along the IMF field lines. The PC module, unique to the UMASEP-10 version, compares the characteristics of the current event to similar historical events to determine if it is more likely to surpass the SPE threshold or return to background. The PC module is intended to detect an event when there is no magnetic connection between the parent event and the observer. The preliminary warnings from both the WC and PC modules are passed to the Analysis and Inference module with preference assigned to the WC module; if this preliminary result is accepted, an alert is issued. UMASEP-10 has recently been updated to include the University of Malaga predictor from Solar Data (UMASOD) model, which incorporates radio data (Zucca et al., 2017; Núñez and Paul-Pena, 2020) to enhance the model's ability to provide advance warning of an SEP event.

UMASEP-100 performs in a similar manner to the WC module, but uses a derived instead of real-value time series. The updated algorithm performs a bit-based transformation of the X-ray flux and the GOES differential proton channels P6 to P11, with a focus on strong positive derivatives (1 vs 0). The newer UMASEP-30 and UMASEP-50 modules were developed based on a similar theory as the original UMASEP-10 and UMASEP-100.

UMASEP-500 is very similar to the UMASEP-100 method, with appropriate changes in the model parameters and boundary conditions. For the model developer's work with the HESPERIA/HORIZON 2020 project, the focus of the work transitioned to GLEs to complement the GLE Alert Plus Tool (Souvatzoglou et al., 2014). For this version, the model developer has also considered replacing the SXR data input stream with microwave flux density

(Hard X-ray (HXR)); however, this functionality is not available real-time.

**Inputs:** All data streams available from NOAA/SWPC:

* GOES SXR

* GOES differential proton flux

* Radio burst data (UMASOD only) from the USAF RSTN

**Outputs:** Viewer displays real-time forecast, including.

* Observed integral proton flux

* Forecasted integral proton flux

* All-clear period (if applicable)

* Observed X-ray flux

* Magnetic connectivity estimation (proportion representing low/medium/high)

* Real-time forecast

* Model inferences in real-time (includes AR information, if available, from SWPC database)

* Early SEP event fluence projection

**Free parameters:** None.

**Limitations and caveats:** Since the WC UMASEP module algorithm emphasizes connectivity, it is unable to predict behind-limb events. As the PC module only analyzes the evolution of in situ protons, this module is able to predict this type of event. Additionally, to limit false alarms, a flare intensity threshold is integrated into each model version; below this threshold, UMASEP does not run. It is also important for the user to understand that the event intensity provided by the model is intended to be the maximum value during the initial 'x' hours of an event, where 'x' varies by model variation.

**Access to model output or forecasts:** Real-time SEP predictions from UMASEP-10, UMASEP-30, UMASEP-50, UMASEP-100 and UMASEP-500 are available on the SEP Scoreboard (https://sep.ccmc.gsfc.nasa.gov/intensity/) and the ISWA interface hosted by CCMC. UMASEP-10 outputs are also available from the University of Malaga. UMASEP-500 outputs are available through the HESPERIA collaboration.

**Model validation:** The model developer used events from SC23 and 24 to validate the results of UMASEP-100 and UMASEP-500; a recent publication detailing updates to UMASEP-10 incorporates solar cycle 22 as well (Table 9). For the UMASEP-10 WC module and the UMASEP-100

<caption>Table 9</caption>

Validation of UMASEP model as performed by developer. UMASEP-10 was validated for predicting all >10 MeV SEP events from 1996 to 2017. UMASEP-100 was validated for predicting all >100 MeV SEP events from 1994 to 2013. UMASEP-500 was validated for predicting all GLE events from 2000 to 2016. POD = Probability of Detection; FAR = False Alarm Rate; AWT = Average Warning Time.

<table><thead><tr><th></th><th>UMASEP-10 (Núñez, 2022)</th><th>UMASEP-100 (Núñez, 2015)</th><th>UMASEP-500 (Núñez et al., 2017)</th></tr></thead><tbody><tr><td>POD (%)</td><td>82.16</td><td>80.85</td><td>53.8</td></tr><tr><td>FAR (%)</td><td>21.52</td><td>29.62</td><td>30.0</td></tr><tr><td>AWT (hh:mm)</td><td>03:15</td><td>01:06</td><td>00:08</td></tr></tbody></table>
100 (Núñez, 2015) predicts >100 MeV events; and UMASEP-500 (Núñez et al., 2017) predicts the occurrence of a GLE. UMASEP-500 was developed in collaboration with the European Unions's HESPERIA project, which also includes the RELEASE code (Section 3.21). Two additional modules, UMASEP-30 and UMASEP-50, have recently been developed to forecast >30 and >50 MeV events. All model versions use similar underlying algorithms; however, the implementation differs.

The preliminary forecast functionality of UMASEP-10 was originally divided into the Well-Connected (WC) and Poorly-Connected (PC) modules. The WC module assesses the relationship between the time series GOES SXR (X-ray emissions) and an increase in GOES differential proton flux to determine if particles have escaped along the IMF field lines. The PC module, unique to the UMASEP-10 version, compares the characteristics of the current event to similar historical events to determine if it is more likely to surpass the SPE threshold or return to background. The PC module is intended to detect an event when there is no magnetic connection between the parent event and the observer. The preliminary warnings from both the WC and PC modules are passed to the Analysis and Inference module with preference assigned to the WC module; if this preliminary result is accepted, an alert is issued. UMASEP-10 has recently been updated to include the University of Malaga predictor from Solar Data (UMASOD) model, which incorporates radio data (Zucca et al., 2017; Núñez and Paul-Pena, 2020) to enhance the model's ability to provide advance warning of an SEP event.

UMASEP-100 performs in a similar manner to the WC module, but uses a derived instead of real-value time series. The updated algorithm performs a bit-based transformation of the X-ray flux and the GOES differential proton channels P6 to P11, with a focus on strong positive derivatives (1 vs 0). The newer UMASEP-30 and UMASEP-50 modules were developed based on a similar theory as the original UMASEP-10 and UMASEP-100.

UMASEP-500 is very similar to the UMASEP-100 method, with appropriate changes in the model parameters and boundary conditions. For the model developer's work with the HESPERIA/HORIZON 2020 project, the focus of the work transitioned to GLEs to complement the GLE Alert Plus Tool (Souvatzoglou et al., 2014). For this version, the model developer has also considered replacing the SXR data input stream with microwave flux density

(Hard X-ray (HXR)); however, this functionality is not available real-time.

**Inputs:** All data streams available from NOAA/SWPC:

* GOES SXR

* GOES differential proton flux

* Radio burst data (UMASOD only) from the USAF RSTN

**Outputs:** Viewer displays real-time forecast, including.

* Observed integral proton flux

* Forecasted integral proton flux

* All-clear period (if applicable)

* Observed X-ray flux

* Magnetic connectivity estimation (proportion representing low/medium/high)

* Real-time forecast

* Model inferences in real-time (includes AR information, if available, from SWPC database)

* Early SEP event fluence projection

**Free parameters:** None.

**Limitations and caveats:** Since the WC UMASEP module algorithm emphasizes connectivity, it is unable to predict behind-limb events. As the PC module only analyzes the evolution of in situ protons, this module is able to predict this type of event. Additionally, to limit false alarms, a flare intensity threshold is integrated into each model version; below this threshold, UMASEP does not run. It is also important for the user to understand that the event intensity provided by the model is intended to be the maximum value during the initial 'x' hours of an event, where 'x' varies by model variation.

**Access to model output or forecasts:** Real-time SEP predictions from UMASEP-10, UMASEP-30, UMASEP-50, UMASEP-100 and UMASEP-500 are available on the SEP Scoreboard (https://sep.ccmc.gsfc.nasa.gov/intensity/) and the ISWA interface hosted by CCMC. UMASEP-10 outputs are also available from the University of Malaga. UMASEP-500 outputs are available through the HESPERIA collaboration.

**Model validation:** The model developer used events from SC23 and 24 to validate the results of UMASEP-100 and UMASEP-500; a recent publication detailing updates to UMASEP-10 incorporates solar cycle 22 as well (Table 9). For the UMASEP-10 WC module and the UMASEP-100

<caption>Table 9</caption>

Validation of UMASEP model as performed by developer. UMASEP-10 was validated for predicting all >10 MeV SEP events from 1996 to 2017. UMASEP-100 was validated for predicting all >100 MeV SEP events from 1994 to 2013. UMASEP-500 was validated for predicting all GLE events from 2000 to 2016. POD = Probability of Detection; FAR = False Alarm Rate; AWT = Average Warning Time.

<table><thead><tr><th></th><th>UMASEP-10 (Núñez, 2022)</th><th>UMASEP-100 (Núñez, 2015)</th><th>UMASEP-500 (Núñez et al., 2017)</th></tr></thead><tbody><tr><td>POD (%)</td><td>82.16</td><td>80.85</td><td>53.8</td></tr><tr><td>FAR (%)</td><td>21.52</td><td>29.62</td><td>30.0</td></tr><tr><td>AWT (hh:mm)</td><td>03:15</td><td>01:06</td><td>00:08</td></tr></tbody></table>
variation, the POD was incremented and the FAR was minimized by optimizing parameters to maximize forecasting performance (Davis and Goadrich, 2006):

$$w_{precision} * Precision + w_{recall} * Recall \qquad (19)$$

where $Recall = POD$ and $Precision = 1 - FAR$. $w_{precision}$ and $w_{recall}$ are weights that are set at 0.5 to give both calculations equal priority. For the UMASEP-10 PC variation, the weights (contributions, or “votes”) of the individual model trees were selected to maximize Precision and Recall, as defined above. The Analysis and Interference Module then selected the thresholds $τ_1$ and $τ_2$, representing temporal limits of the event, to maximize forecasting performance as defined above. For the UMASEP-500 variation, a successful forecast was defined as whether the model alert preceded neutron monitor alert. A low POD indicated no sign of proton enhancement before neutron monitor alert. The HESPERIA UMASEP-500 tool makes real-time predictions of the occurrence of >500 MeV and GLE events from the analysis of soft X-ray flux and high-energy differential proton flux measured by the GOES satellite network. Regarding the prediction of GLE events for the period 2000–2016, this tool had a POD of 53.8% and a FAR of 30.0%. For this period, the tool obtained an Average Warning Time (AWT) of 8 min taking as reference the alert time from the first neutron monitor station; using the time of the warning issued by the GLE Alert Plus tool for the aforementioned period as reference, the tool obtained an AWT of 15 min.

### 3.36. Zhang Model

**Model developers:** Ming Zhang (Florida Institute of Technology).

**Model description:** The Zhang model is a physics-based SEP model taking a focused transport approach to predicting SEP events. In Zhang and Zhao (2017), the developers use a PFSS model to set up the coronal magnetic field, where NSO GONG’s spherical harmonic coefficients are fit to photospheric magnetogram measurements. The model uses the first available coefficient post-eruption, since CMEs can significantly alter the coronal magnetic field configuration. Above the source surface ($R_{ss} > 2.5R_{s}$), the solar wind speed and density is modelled after Leblanc et al. (1998) and the magnetic field in the solar wind is a Parker spiral. These become initial conditions to constrain the solution to the transport side of the model. The FTE is a 3-D time-dependent equation with terms for pitch-angle diffusion, magnetic focusing, adiabatic cooling and pitch angle change, particle streaming, perpendicular diffusion, convection with the solar wind, particle drift via gradient and curvature drifts, and particle injection rate. Perpendicular diffusion is modelled after the random walk of field lines, and contributes to the longitudinal spread of particles. From the model developer’s studies, see Zhang and Zhao (2017), Zhao et al. (2020), the value for the perpendicular diffusion can change event-

by-event, but a typical value is ~10% of the supergranular diffusion rate. The pitch angle diffusion coefficient takes a form from quasi-linear theory, where some nonlinear corrections have been made. Since perpendicular diffusion is included, the FTE becomes a time-dependent parabolic partial differential equation in 5-D phase space that can be solved via a Stochastic Differential Equation (SDE). This model can solve the SDE either in a time-forwards or time-backwards scheme, dependent upon whichever one minimizes computation time.

**Inputs:**

* solar wind speed

* perpendicular diffusion coefficient

* outer boundary (50 AU)

* particle injection profile

* particle mean free path

* For PFSS: spherical harmonic coefficients derived from synoptic magnetograms

**Outputs:**

* Time intensity profile

* SEP distributions

**Free Parameters:** None.

**Limitations and caveats:** Since this model takes advantage of the PFSS, it’s coronal modelling starts deeper down to the solar surface within the corona than other similar models; this also contributes to how accurately the model can capture the magnetic connection to the source location within the corona. However, this comes with the downsides of PFSS, where small scale structures are not modelled well, which shouldn’t affect the overall predictive qualities of the Zhang model as particle transport is not greatly affected by those structures (Zhao and Zhang, 2018).

**Access to model output or forecasts:** There is currently no public access to model output.

**Model validation:** This model has been used for singular events as a research model (see Zhang and Zhao, 2017; Zhao and Zhang, 2018; Zhao et al., 2020 for the most recent case studies). Many of these studies involve looking at particle distributions in the corona and other underlying physics, not comparisons to observations - so no validation studies have been published.

### 4. Conclusions

This paper has summarized the current SEP models produced by the scientific community. Section 3 describes a wide range of models that have been developed for various purposes, often not specifically for SEP prediction, and are in different stages of maturity. Some well-established SEP models are actively providing forecasts in real time in support of operations while others, such as ML models, are pursuing new approaches as a proof of concept. Physics-based SEP models are increasing in sophistication and
variation, the POD was incremented and the FAR was minimized by optimizing parameters to maximize forecasting performance (Davis and Goadrich, 2006):

$$w_{precision} * Precision + w_{recall} * Recall \quad (19)$$

where $Recall = POD$ and $Precision = 1 - FAR$. $w_{precision}$ and $w_{recall}$ are weights that are set at 0.5 to give both calculations equal priority. For the UMASEP-10 PC variation, the weights (contributions, or “votes”) of the individual model trees were selected to maximize Precision and Recall, as defined above. The Analysis and Interference Module then selected the thresholds $τ_1$ and $τ_2$, representing temporal limits of the event, to maximize forecasting performance as defined above. For the UMASEP-500 variation, a successful forecast was defined as whether the model alert preceded neutron monitor alert. A low POD indicated no sign of proton enhancement before neutron monitor alert. The HESPERIA UMASEP-500 tool makes real-time predictions of the occurrence of >500 MeV and GLE events from the analysis of soft X-ray flux and high-energy differential proton flux measured by the GOES satellite network. Regarding the prediction of GLE events for the period 2000–2016, this tool had a POD of 53.8% and a FAR of 30.0%. For this period, the tool obtained an Average Warning Time (AWT) of 8 min taking as reference the alert time from the first neutron monitor station; using the time of the warning issued by the GLE Alert Plus tool for the aforementioned period as reference, the tool obtained an AWT of 15 min.

### 3.36. Zhang Model

**Model developers:** Ming Zhang (Florida Institute of Technology).

**Model description:** The Zhang model is a physics-based SEP model taking a focused transport approach to predicting SEP events. In Zhang and Zhao (2017), the developers use a PFSS model to set up the coronal magnetic field, where NSO GONG’s spherical harmonic coefficients are fit to photospheric magnetogram measurements. The model uses the first available coefficient post-eruption, since CMEs can significantly alter the coronal magnetic field configuration. Above the source surface ($R_{ss} > 2.5R_{s}$), the solar wind speed and density is modelled after Leblanc et al. (1998) and the magnetic field in the solar wind is a Parker spiral. These become initial conditions to constrain the solution to the transport side of the model. The FTE is a 3-D time-dependent equation with terms for pitch-angle diffusion, magnetic focusing, adiabatic cooling and pitch angle change, particle streaming, perpendicular diffusion, convection with the solar wind, particle drift via gradient and curvature drifts, and particle injection rate. Perpendicular diffusion is modelled after the random walk of field lines, and contributes to the longitudinal spread of particles. From the model developer’s studies, see Zhang and Zhao (2017), Zhao et al. (2020), the value for the perpendicular diffusion can change event-

by-event, but a typical value is ~10% of the supergranular diffusion rate. The pitch angle diffusion coefficient takes a form from quasi-linear theory, where some nonlinear corrections have been made. Since perpendicular diffusion is included, the FTE becomes a time-dependent parabolic partial differential equation in 5-D phase space that can be solved via a Stochastic Differential Equation (SDE). This model can solve the SDE either in a time-forwards or time-backwards scheme, dependent upon whichever one minimizes computation time.

**Inputs:**

* solar wind speed

* perpendicular diffusion coefficient

* outer boundary (50 AU)

* particle injection profile

* particle mean free path

* For PFSS: spherical harmonic coefficients derived from synoptic magnetograms

**Outputs:**

* Time intensity profile

* SEP distributions

**Free Parameters:** None.

**Limitations and caveats:** Since this model takes advantage of the PFSS, it’s coronal modelling starts deeper down to the solar surface within the corona than other similar models; this also contributes to how accurately the model can capture the magnetic connection to the source location within the corona. However, this comes with the downsides of PFSS, where small scale structures are not modelled well, which shouldn’t affect the overall predictive qualities of the Zhang model as particle transport is not greatly affected by those structures (Zhao and Zhang, 2018).

**Access to model output or forecasts:** There is currently no public access to model output.

**Model validation:** This model has been used for singular events as a research model (see Zhang and Zhao, 2017; Zhao and Zhang, 2018; Zhao et al., 2020 for the most recent case studies). Many of these studies involve looking at particle distributions in the corona and other underlying physics, not comparisons to observations - so no validation studies have been published.

## 4. Conclusions

This paper has summarized the current SEP models produced by the scientific community. Section 3 describes a wide range of models that have been developed for various purposes, often not specifically for SEP prediction, and are in different stages of maturity. Some well-established SEP models are actively providing forecasts in real time in support of operations while others, such as ML models, are pursuing new approaches as a proof of concept. Physics-based SEP models are increasing in sophistication and
<table><caption>Table 10
Observational measurements used as inputs into SEP models.</caption><thead><tr><th>Model</th><th>Type</th><th>Magnetograms</th><th>Optical Imaging</th><th>EUV Imaging</th><th>Soft X-ray Intensity</th><th>Ground-based Radio</th><th>Space-based Radio</th><th>Coronagraph</th><th>Solar Wind (n,T,p,v)</th><th>Suprathelial Particles</th><th>Energetic Protons</th><th>Energetic Electrons</th><th>Neutron Monitors</th></tr></thead><tbody><tr><td>ADEPT</td><td>Empirical</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>AFRL PPS</td><td>Empirical</td><td></td><td>x</td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>Aminalragia-Giamini model</td><td>ML</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>AMPS</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Boubrahimi model</td><td>ML</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>COMESEP SEPForecast</td><td>Emp. &amp; Physics</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td>x</td><td></td><td>x</td><td></td><td></td><td>x</td></tr><tr><td>EPREM</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>ESPERTA</td><td>Emp. &amp; ML</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>FORSPEF</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>GSU</td><td>ML</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>iPATH</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td></tr><tr><td>Lavasa Model</td><td>ML</td><td></td><td>x</td><td></td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MAG4</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MagPy</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MEMPSEP</td><td>ML</td><td>x</td><td></td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>M-FLAMPA</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PARADISE</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PCA model</td><td>Empirical</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PHSVM</td><td>ML</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>PROTONS</td><td>Empirical</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>REleASE</td><td>Empirical</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td></tr><tr><td>Sadykov's Model</td><td>ML</td><td>x</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>SAWS-ASPECS</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>SEPCaster</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>SEPMOD</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SEPSTER</td><td>Empirical</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>SEPSTER2D</td><td>Empirical</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>SMARP Model</td><td>ML</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SOLPENCO</td><td>Physics-based</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SOLPENCO(2)</td><td>Physics-based</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td></tr><tr><td>South African model</td><td>Physics-based</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SPARX</td><td>Physics-based</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SPREADFAST</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>SPRINTS</td><td>ML</td><td>x</td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>STAT</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td>x</td><td></td><td></td><td></td></tr><tr><td>UMASEP</td><td>Empirical</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>Zhang model</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>Total</td><td></td><td>19</td><td>6</td><td>21</td><td>19</td><td>4</td><td>3</td><td>21</td><td>7</td><td>5</td><td>11</td><td>3</td><td>2</td></tr></tbody></table>

aim to partially or fully model the heliophysics system, coupling coronal, heliospheric, and particle acceleration/-transport models together. The model summaries explicitly identify the observational inputs for each model, demonstrating the range of critical observational infrastructure that is required for effective SEP forecasting. Models generally have limitations as well as caveats that should be understood to accurately interpret model output. Finally,

model validation efforts are summarized in order to understand the current state of the art of SEP model performance as well as to emphasize that all models and model types can benefit from thorough, quantitative validation.

In the remaining sections, we will draw conclusions from this comprehensive look at the current state of SEP modeling. We will highlight the critical measurements used by models, describe forecasting coverage and identify any
<table><caption>Table 10
Observational measurements used as inputs into SEP models.</caption><thead><tr><th>Model</th><th>Type</th><th>Magnetograms</th><th>Optical Imaging</th><th>EUV Imaging</th><th>Soft X-ray Intensity</th><th>Ground-based Radio</th><th>Space-based Radio</th><th>Coronagraph</th><th>Solar Wind (n,T,p,v)</th><th>Suprathelial Particles</th><th>Energetic Protons</th><th>Energetic Electrons</th><th>Neutron Monitors</th></tr></thead><tbody><tr><td>ADEPT</td><td>Empirical</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>AFRL PPS</td><td>Empirical</td><td></td><td>x</td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Aminalragia-Giamini model</td><td>ML</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>AMPS</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Boubrahimi model</td><td>ML</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>COMESEP SEPForecast</td><td>Emp. &amp; Physics</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td>x</td><td></td><td>x</td><td></td><td></td><td>x</td></tr><tr><td>EPREM</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td>x</td><td></td><td></td><td></td></tr><tr><td>ESPERTA</td><td>Emp. &amp; ML</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>FORSPEF</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>GSU</td><td>ML</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>iPATH</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td></tr><tr><td>Lavasa Model</td><td>ML</td><td></td><td>x</td><td></td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MAG4</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MagPy</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MEMPSEP</td><td>ML</td><td>x</td><td></td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td></tr><tr><td>M-FLAMPA</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PARADISE</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PCA model</td><td>Empirical</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PHSVM</td><td>ML</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td></tr><tr><td>PROTONS</td><td>Empirical</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>REleASE</td><td>Empirical</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td></tr><tr><td>Sadykov's Model</td><td>ML</td><td>x</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>SAWS-ASPECS</td><td>Empirical</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>SEPCaster</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>SEPMOD</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SEPSTER</td><td>Empirical</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>SEPSTER2D</td><td>Empirical</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>SMARP Model</td><td>ML</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SOLPENCO</td><td>Physics-based</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SOLPENCO(2)</td><td>Physics-based</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td></tr><tr><td>South African model</td><td>Physics-based</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SPARX</td><td>Physics-based</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SPREADFAST</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>SPRINTS</td><td>ML</td><td>x</td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>STAT</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td></td><td>x</td><td></td><td></td><td></td></tr><tr><td>UMASEP</td><td>Empirical</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td>x</td><td></td><td></td></tr><tr><td>Zhang model</td><td>Physics-based</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td></tr><tr><td>Total</td><td></td><td>19</td><td>6</td><td>21</td><td>19</td><td>4</td><td>3</td><td>21</td><td>7</td><td>5</td><td>11</td><td>3</td><td>2</td></tr></tbody></table>

aim to partially or fully model the heliophysics system, coupling coronal, heliospheric, and particle acceleration/-transport models together. The model summaries explicitly identify the observational inputs for each model, demonstrating the range of critical observational infrastructure that is required for effective SEP forecasting. Models generally have limitations as well as caveats that should be understood to accurately interpret model output. Finally,

model validation efforts are summarized in order to understand the current state of the art of SEP model performance as well as to emphasize that all models and model types can benefit from thorough, quantitative validation.

In the remaining sections, we will draw conclusions from this comprehensive look at the current state of SEP modeling. We will highlight the critical measurements used by models, describe forecasting coverage and identify any
gaps, highlight models already running in real time, and discuss what is needed to ensure all models are appropriately validated.

## 4.1. Critical measurements

In Table 10, the observations required for input into each model are specified. The bottom row shows the total number of models that use each kind of observation. For models that ingest CME parameters produced by forecasters or extracted from a database, we specify the observations required to produce these parameters. CME parameters are derived using as many coronagraph vantage points as possible, as well as EUV imagery from the SDO or STEREO to view the eruption source region (Millward et al., 2013). For models that use output from another model to run, we identify the observations required as input into the first model in the chain. Physics-based models often run on top of MHD solar wind models with propagating CMEs (e.g. WSA-ENLIL + Cone and SEPMOD, MAS + CORHEL and EPREM, AWSOM + EEGGL and MFLAMPA) and these require magnetograms and CME parameters as input. Some models ingest and report NOAA AR information, which is derived from the SOON<fnref n="20" /> by the USAF (Robert Steenburg, NOAA/SWPC forecaster, private communication). H-alpha imagery from SOON is also used to identify solar flare locations. Databases such as the Solar DEMON derive solar flare location from real time EUV imagery from SDO. Other flare databases, such as Hinode/EUV Imaging Spectrometer (EIS)<fnref n="21" /> provide flare locations derived from X-ray imagery, which are valuable for model training but are not available as real time inputs. The GOES-R XRS product on the NOAA website<fnref n="22" /> provides flare location from the XRS quad diodes (Machol et al., 2021). Until recently, this location product was not available in real time. For the models described in this paper, flare locations are derived in real time from H-alpha or EUV imagery, however the GOES XRS locations services may provide an alternative now that they are available.

Table 10 clearly demonstrates the heavy current utilization of magnetograms, EUV imaging, coronagraph observations and energetic particle observations for SEP prediction. Therefore, we emphasize that such data streams must be supported operationally to ensure that SEP models can be used for forecasting – this includes the continuity of existing data streams and development of new ones to enhance or replace them. However, it should be noted that those observations that are less widely used also provide valuable inputs into SEP prediction and should also be supported operationally. In particular, neutron monitors provide confirmation of the onset of an extremely high-

energy SEP event (e.g., He and Rodriguez, 2018), while energetic electrons provide direct observational forewarning of an SEP event (Posner, 2007). Also, currently only a few models are listed as using suprathermal ions as input, but continuing research into characterizing suprathermal ions would also benefit other physics-based models that need to specify a “seed spectrum” as a starting point for acceleration at shocks. The limited use of space-based radio emissions reflects the fact that such observations are not currently available in near-real time, notwithstanding their demonstrated strong association with historical SEP events (e.g., Gopalswamy et al., 2005; Cane et al., 2002; Winter and Ledbetter, 2015; Miteva et al., 2017). There is a tendency for radio emission to be weak or absent when a solar event is not accompanied by a significant SEP event (e.g., Richardson et al., 2018; Lario et al., 2020) and therefore, the presence of strong radio emission has been included as a predictor in some SEP models (e.g., Laurenza et al., 2009; Richardson et al., 2018). As described in Section 2.1, critical radio frequencies are not accessible from the ground, which argues in favor of operational space-based, near-real time, solar radio monitoring capability.

Finally, observations of the type that are most widely utilized for SEP prediction along the Sun-Earth line would have greatly enhanced value if obtained from the L4 Lagrange point, where the observed central meridian lies at W60° (Posner et al., 2021). Magnetograms, EUV imagery, coronagraph images, and in situ particles and fields measurements from this vantage point would cover locations on the Sun with the highest degree of magnetic connectivity to the Earth-Moon system and points along minimum-energy transfer (Hohmann) orbits to Mars. SEP events from these well-connected source locations have the highest potential impact, thus posing the largest risk for human radiation and space hardware concerns. This includes locations that are beyond the Earth–Sun west limb that our present set of observatories do not cover, therefore limiting our ability to produce a complete All Clear forecast.

## 4.2. Model coverage for SEP forecasting

In operations, space weather forecasters are asked to answer the following questions:

* Will there be an SEP event (or can an All Clear be issued)?

* When will it occur?

* How intense will it be?

* How long will it last?

Table 11 identifies the predicted quantities output by the models discussed in this paper and whether each model produces output before (“Pre”) a flare or CME or after (“Post”) the eruptive event. The specific meaning of each field is indicated in the table caption. Most are self-explanatory, but “Flux Point” indicates that a model pre-

<footnote n="20">²⁰ https://www.wbdg.org/FFC/AF/AFMAN/149XX5_Solar_Observing_Optical_Network1.pdf.</footnote>

<footnote n="21">²¹ http://solarb.mssl.ucl.ac.uk/SolarB/eisflare2017.jsp.</footnote>

<footnote n="22">²² https://www.ngdc.noaa.gov/stp/satellite/goes-r.html.</footnote>
gaps, highlight models already running in real time, and discuss what is needed to ensure all models are appropriately validated.

## 4.1. Critical measurements

In Table 10, the observations required for input into each model are specified. The bottom row shows the total number of models that use each kind of observation. For models that ingest CME parameters produced by forecasters or extracted from a database, we specify the observations required to produce these parameters. CME parameters are derived using as many coronagraph vantage points as possible, as well as EUV imagery from the SDO or STEREO to view the eruption source region (Millward et al., 2013). For models that use output from another model to run, we identify the observations required as input into the first model in the chain. Physics-based models often run on top of MHD solar wind models with propagating CMEs (e.g. WSA-ENLIL + Cone and SEPMOD, MAS + CORHEL and EPREM, AWSOM + EEGGL and MFLAMPA) and these require magnetograms and CME parameters as input. Some models ingest and report NOAA AR information, which is derived from the SOON<fnref n="20" /> by the USAF (Robert Steenburg, NOAA/SWPC forecaster, private communication). H-alpha imagery from SOON is also used to identify solar flare locations. Databases such as the Solar DEMON derive solar flare location from real time EUV imagery from SDO. Other flare databases, such as Hinode/EUV Imaging Spectrometer (EIS)<fnref n="21" /> provide flare locations derived from X-ray imagery, which are valuable for model training but are not available as real time inputs. The GOES-R XRS product on the NOAA website<fnref n="22" /> provides flare location from the XRS quad diodes (Machol et al., 2021). Until recently, this location product was not available in real time. For the models described in this paper, flare locations are derived in real time from H-alpha or EUV imagery, however the GOES XRS locations services may provide an alternative now that they are available.

Table 10 clearly demonstrates the heavy current utilization of magnetograms, EUV imaging, coronagraph observations and energetic particle observations for SEP prediction. Therefore, we emphasize that such data streams must be supported operationally to ensure that SEP models can be used for forecasting – this includes the continuity of existing data streams and development of new ones to enhance or replace them. However, it should be noted that those observations that are less widely used also provide valuable inputs into SEP prediction and should also be supported operationally. In particular, neutron monitors provide confirmation of the onset of an extremely high-

energy SEP event (e.g., He and Rodriguez, 2018), while energetic electrons provide direct observational forewarning of an SEP event (Posner, 2007). Also, currently only a few models are listed as using suprathermal ions as input, but continuing research into characterizing suprathermal ions would also benefit other physics-based models that need to specify a “seed spectrum” as a starting point for acceleration at shocks. The limited use of space-based radio emissions reflects the fact that such observations are not currently available in near-real time, notwithstanding their demonstrated strong association with historical SEP events (e.g., Gopalswamy et al., 2005; Cane et al., 2002; Winter and Ledbetter, 2015; Miteva et al., 2017). There is a tendency for radio emission to be weak or absent when a solar event is not accompanied by a significant SEP event (e.g., Richardson et al., 2018; Lario et al., 2020) and therefore, the presence of strong radio emission has been included as a predictor in some SEP models (e.g., Laurenza et al., 2009; Richardson et al., 2018). As described in Section 2.1, critical radio frequencies are not accessible from the ground, which argues in favor of operational space-based, near-real time, solar radio monitoring capability.

Finally, observations of the type that are most widely utilized for SEP prediction along the Sun-Earth line would have greatly enhanced value if obtained from the L4 Lagrange point, where the observed central meridian lies at W60° (Posner et al., 2021). Magnetograms, EUV imagery, coronagraph images, and in situ particles and fields measurements from this vantage point would cover locations on the Sun with the highest degree of magnetic connectivity to the Earth-Moon system and points along minimum-energy transfer (Hohmann) orbits to Mars. SEP events from these well-connected source locations have the highest potential impact, thus posing the largest risk for human radiation and space hardware concerns. This includes locations that are beyond the Earth–Sun west limb that our present set of observatories do not cover, therefore limiting our ability to produce a complete All Clear forecast.

## 4.2. Model coverage for SEP forecasting

In operations, space weather forecasters are asked to answer the following questions:

* Will there be an SEP event (or can an All Clear be issued)?

* When will it occur?

* How intense will it be?

* How long will it last?

Table 11 identifies the predicted quantities output by the models discussed in this paper and whether each model produces output before (“Pre”) a flare or CME or after (“Post”) the eruptive event. The specific meaning of each field is indicated in the table caption. Most are self-explanatory, but “Flux Point” indicates that a model pre-

<footnote n="20">²⁰ https://www.wbdg.org/FFC/AF/AFMAN/149XX5_Solar_Observering_Optical_Network1.pdf.</footnote>

<footnote n="21">²¹ http://solarb.mssl.ucl.ac.uk/SolarB/eisflare2017.jsp.</footnote>

<footnote n="22">²² https://www.ngdc.noaa.gov/stp/satellite/goes-r.html.</footnote>
<caption>Table 11</caption>

Outputs produced by the SEP models summarized in this paper. Pre/Post: Pre indicates pre-eruptive forecast prior to the flare or CME, Post indicates a forecast issued after an eruptive event (flare, CME) has occurred; All Clear: binary yes/no forecast for an SEP event or specific threshold crossing; Probability: probability of occurrence; Flux Point: forecast of proton intensity levels for a single time point or a single flux value within a specific time window in the future (see main text for further description); Onset time: time of threshold crossing or SEP event start; Peak: peak intensity; Peak time: time of the peak intensity; End time: event end time or decay time; Fluence: total event time-integrated fluence; Time profile: produces intensity with time; Multi loc.: capable of producing forecasts for multiple locations in the heliosphere; 3D: produces 3D environmental data and particle info, such as pitch angle distributions.

<table><thead><tr><th>Model</th><th>Proton Energy [MeV]</th><th>Pre/Post</th><th>All Clear</th><th>Probability</th><th>Flux Point</th><th>Onset time</th><th>Peak</th><th>Peak time</th><th>End time</th><th>Fluence</th><th>Time profile</th><th>Multi loc.</th><th>3D</th></tr></thead><tbody><tr><td>ADEPT</td><td>&gt;10, &gt;30, &gt;50, &gt;100</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>AFRL PPS</td><td>&gt;5, &gt;10, &gt;50</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td></tr><tr><td>Aminalragia-Giamini model</td><td>≥5</td><td>Post</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>AMPS</td><td>eV to GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>Boubrahimi model</td><td>&gt;100</td><td>Post</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>COMESEP SEPForecast</td><td>&gt;10, &gt;60</td><td>Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td></tr><tr><td>EPREM</td><td>5 - 1000**</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>ESPERTA</td><td>&gt;10</td><td>Post</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td></tr><tr><td>FORSPEF</td><td>&gt;10, &gt;30, &gt;60, &gt;100</td><td>Pre/Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>GSU</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>iPATH</td><td>100 keV - GeV</td><td>Post</td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>Lavasa Model</td><td>&gt;10</td><td>Pre</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MAG4</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MagPy</td><td>&gt;10, &gt;100</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MEMPSEP</td><td>9-15, &gt;5, &gt;10, &gt;30, &gt;60, &gt;100</td><td>Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>M-FLAMPA</td><td>10 keV - 1 GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>PARADISE</td><td>keV - GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>PCA model</td><td>&gt;10</td><td>Post</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PHSVM</td><td>&gt;100</td><td>Post</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PROTONS</td><td>&gt;10</td><td>Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>REeASE</td><td>4-9; 9-15.8; 15.8-39.8; 28.2-50.1</td><td>Post</td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Sadykov et al.</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SAWS-ASPECS</td><td>&gt;10 to &gt;300</td><td>Pre/Post</td><td>x</td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td></tr><tr><td>SEPCaster</td><td>100 keV - GeV</td><td>Post</td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>SEPMOD</td><td>1 - 1000</td><td>Post</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>x</td></tr><tr><td>SEPSTER</td><td>14 - 24; &gt;10, &gt;30, &gt;50, &gt;100</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>SEPSTER2D</td><td>10 - 130; &gt;130</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td></td></tr><tr><td>SMARP Model</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SOLPENCO(2)</td><td>0.125 - 64; 5 - 300</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td></tr><tr><td>South African model</td><td>keV - GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>SPARX</td><td>&gt;10, &gt;60, &gt;300</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>SPREAdFAST</td><td>2 - 115</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>SPRINTS</td><td>1, 5, 10, 30, 50, 100</td><td>Pre/Post</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>STAT</td><td>1 - 1000</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>UMASEP</td><td>&gt;10, &gt;30, &gt;50, &gt;100, &gt;500</td><td>Post</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td></tr><tr><td>Zhang model</td><td>MeV - GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr></tbody></table>

dicts the particle flux for a specific time or time range into the future. For example, UMASEP forecasts the maximum flux in the next 3 to 24 h (depending on energy channel and magnetic connectivity) and this forecasted flux value may not necessarily correspond to the peak flux, which could occur outside of the time window for very gradual events. RELEASE forecasts the differential proton intensity 30,

60, or 90 min in the future. The “3D” label indicates that a wide range of information calculated by many physics-based models describing the energetic particle distributions and magnetic field environment in 3-dimensional space, such as pitch-angle distributions and magnetic field line locations, is provided. If a model outputs a time profile, then it is indicated that the model predicts onset time, peak
<caption>Table 11</caption>

Outputs produced by the SEP models summarized in this paper. Pre/Post: Pre indicates pre-eruptive forecast prior to the flare or CME, Post indicates a forecast issued after an eruptive event (flare, CME) has occurred; All Clear: binary yes/no forecast for an SEP event or specific threshold crossing; Probability: probability of occurrence; Flux Point: forecast of proton intensity levels for a single time point or a single flux value within a specific time window in the future (see main text for further description); Onset time: time of threshold crossing or SEP event start; Peak: peak intensity; Peak time: time of the peak intensity; End time: event end time or decay time; Fluence: total event time-integrated fluence; Time profile: produces intensity with time; Multi loc.: capable of producing forecasts for multiple locations in the heliosphere; 3D: produces 3D environmental data and particle info, such as pitch angle distributions.

<table><thead><tr><th>Model</th><th>Proton Energy [MeV]</th><th>Pre/Post</th><th>All Clear</th><th>Probability</th><th>Flux Point</th><th>Onset time</th><th>Peak</th><th>Peak time</th><th>End time</th><th>Fluence</th><th>Time profile</th><th>Multi loc.</th><th>3D</th></tr></thead><tbody><tr><td>ADEPT</td><td>&gt;10, &gt;30, &gt;50, &gt;100</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>AFRL PPS</td><td>&gt;5, &gt;10, &gt;50</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td></tr><tr><td>Aminalragia-Giamini model</td><td>≥5</td><td>Post</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>AMPS</td><td>eV to GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>Boubrahimi model</td><td>&gt;100</td><td>Post</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>COMESEP SEPForecast</td><td>&gt;10, &gt;60</td><td>Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td><td></td></tr><tr><td>EPREM</td><td>5 - 1000**</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>ESPERTA</td><td>&gt;10</td><td>Post</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>FORSPEF</td><td>&gt;10, &gt;30, &gt;60, &gt;100</td><td>Pre/Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>GSU</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>iPATH</td><td>100 keV - GeV</td><td>Post</td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>Lavasa Model</td><td>&gt;10</td><td>Pre</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MAG4</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MagPy</td><td>&gt;10, &gt;100</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>MEMPSEP</td><td>9-15, &gt;5, &gt;10, &gt;30, &gt;60, &gt;100</td><td>Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td></td></tr><tr><td>M-FLAMPA</td><td>10 keV - 1 GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>PARADISE</td><td>keV - GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>PCA model</td><td>&gt;10</td><td>Post</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PHSVM</td><td>&gt;100</td><td>Post</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>PROTONS</td><td>&gt;10</td><td>Post</td><td></td><td>x</td><td></td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>REleASE</td><td>4-9; 9-15.8; 15.8-39.8; 28.2-50.1</td><td>Post</td><td></td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Sadykov et al.</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SAWS-ASPECS</td><td>&gt;10 to &gt;300</td><td>Pre/Post</td><td>x</td><td>x</td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td></tr><tr><td>SEPCaster</td><td>100 keV - GeV</td><td>Post</td><td>x</td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>SEPMOD</td><td>1 - 1000</td><td>Post</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SEPSTER</td><td>14 - 24; &gt;10, &gt;30, &gt;50, &gt;100</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>SEPSTER2D</td><td>10 - 130; &gt;130</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td></td></tr><tr><td>SMARP Model</td><td>&gt;10</td><td>Pre</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>SOLPENCO(2)</td><td>0.125 - 64; 5 - 300</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td></td></tr><tr><td>South African model</td><td>keV - GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>SPARX</td><td>&gt;10, &gt;60, &gt;300</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr><tr><td>SPREAdFAST</td><td>2 - 115</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>SPRINTS</td><td>1, 5, 10, 30, 50, 100</td><td>Pre/Post</td><td>x</td><td>x</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>STAT</td><td>1 - 1000</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td></tr><tr><td>UMASEP</td><td>&gt;10, &gt;30, &gt;50, &gt;100, &gt;500</td><td>Post</td><td>x</td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td></td><td>x</td><td></td><td></td><td></td></tr><tr><td>Zhang model</td><td>MeV - GeV</td><td>Post</td><td></td><td></td><td></td><td></td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td><td>x</td></tr></tbody></table>

dicts the particle flux for a specific time or time range into the future. For example, UMASEP forecasts the maximum flux in the next 3 to 24 h (depending on energy channel and magnetic connectivity) and this forecasted flux value may not necessarily correspond to the peak flux, which could occur outside of the time window for very gradual events. RELEASE forecasts the differential proton intensity 30,

60, or 90 min in the future. The “3D” label indicates that a wide range of information calculated by many physics-based models describing the energetic particle distributions and magnetic field environment in 3-dimensional space, such as pitch-angle distributions and magnetic field line locations, is provided. If a model outputs a time profile, then it is indicated that the model predicts onset time, peak
flux and time, end time, and fluence as applicable. There are some time profile models that cannot currently simulate the full duration of the event and for these, only the predictions that are possible to derive from their time profiles are indicated.

It is important to consider whether the model outputs together can answer all the above questions or if there are clear gaps in our current capabilities. All 35 models generate predictions for protons in the ~10 MeV energy range. This is not surprising as the NOAA definition of an SEP event is >10 MeV proton flux exceeds 10 pfu, and modelers have focused on providing predictions that are consistent with this definition. In terms of human space exploration, >10 MeV particles pose a danger for astronauts outside of a vehicle during an EVA in space or on the lunar surface. Higher energy particles that can penetrate spacecraft shielding are a danger for astronauts throughout a mission, particularly exploration missions outside of Low-Earth Orbit (LEO). SRAG at NASA/JSC applies a threshold of >100 MeV proton flux exceeding 1 pfu to indicate when the particle environment is of concern. About half of the models provide predictions in that energy range, but many of those are physics-based models that face challenges in running in real time. Developing (and validating) empirical and machine learning models to make high energy proton predictions is also a challenge since SEP events that reach such energies are relatively rare. From a space radiation operations perspective, Table 11 demonstrates a need to expand forecasting to higher energies, particularly by “fast and light” models that can produce results ahead of an eruption or quickly following an eruptive event.

Table 10 shows that many of the models rely on flare or CME information to make their predictions, thus it is not surprising that most models in Table 11 provide post-eruption forecasts. The models that do make a prediction ahead of an eruption typically provide an All Clear and/or probability of occurrence. These models rely on information derived from magnetograms and take empirical or machine learning approaches. Indeed, pre-eruptive forecasting capabilities may be an area where machine learning algorithms can make advances over more traditional empirical models. This will remain to be seen until the application of ML to space weather forecasting matures.

Table 10 also indicates the current limited use of energetic electrons as model input and identifies an opportunity for future models to take advantage of their potential to provide an early warning of a proton event or as an effective discriminator to reduce false alarms. The detection of energetic electrons may be particularly important for space exploration missions as external electron detectors could be mounted on board the vehicle to create an early warning system at the spacecraft. Posner et al. (2020) have shown that electron measurements at Earth-L1 and Mars-L1 used with the RELEASE forecasting system could provide advanced warning of proton events throughout most of a mission to Mars.

Overall, Table 11 shows that the models, taken as a whole, provide good coverage of all listed quantities with high redundancy that could be used to address the above forecaster questions. However, a major current restriction is that most of the model outputs are not available to forecasters in real time. In some cases, this is due to the latency in observations, e.g., coronagraph images of CMEs. In other cases, models are not yet robust enough to run continuously or cannot produce output faster than real time. Space agencies around the world have developed forecasting dashboards for individual or a small number of SEP models, but the opportunity remains to make use of the diverse set of models that exist within the research community. Increased Research-to Operations (R2O) efforts are needed to prepare and onboard SEP models into operational settings.

### 4.3. Models running operationally or in real time

Significant effort is required to move a model from the development phase to running robustly in real time. Twelve of the SEP models discussed in this paper have undertaken these steps and are running in a real time or operational environment. Here, operational models are defined as those being served through the ESA Space Weather Service Network (https://swe.ssa.esa.int/) for its space weather users or utilized by the USAF or NOAA SWPC for space weather forecasting. These models have undergone a vetting process for robustness, stability, and performance by their respective operational agencies and have external users that monitor their forecasts for decision-making purposes. Five models satisfy these criteria: AFRL PPS, COMESEP<fnref n="23" />, PROTONS, RELEASE, and UMASEP.

An additional seven models are defined as running in real time: FORSPEF<fnref n="24" />, MAG4, SAWS-ASPECS<fnref n="25" />, SEP-MOD, SEPSTER, SEPSTER2D, and SPRINTS. These models have done the key work to ensure stability and are deployed on public-facing environments, like the applications developed at CCMC. Model forecasts are monitored by users on an experimental basis.

The ESA Space Weather Service Network provides a dashboard of forecasts to support a wide variety of end-users in order to mitigate the effects of space weather on their systems (Kruglanski et al., 2016). The Network is organized around five Expert Service Centres that each have an individual focus on Solar Weather, Heliospheric Weather, Space Radiation Environment, Ionospheric Weather and Geomagnetic Conditions.

The USAF Space Weather Operations Center (SpWOC) is the only Department of Defense (DoD) unit delivering 24/7 space weather information for the Total Force,

<footnote n="23">²³ SPARX predictions are used in COMESEP forecasting in the form of a pre-run database.</footnote>

<footnote n="24">²⁴ The PCA Model provides probabilistic forecasts within FORSPEF.</footnote>

<footnote n="25">²⁵ SOLPENCO2 predictions are used in SAWS-ASPECS forecasting in the form of a pre-run database.</footnote>
flux and time, end time, and fluence as applicable. There are some time profile models that cannot currently simulate the full duration of the event and for these, only the predictions that are possible to derive from their time profiles are indicated.

It is important to consider whether the model outputs together can answer all the above questions or if there are clear gaps in our current capabilities. All 35 models generate predictions for protons in the ~10 MeV energy range. This is not surprising as the NOAA definition of an SEP event is >10 MeV proton flux exceeds 10 pfu, and modelers have focused on providing predictions that are consistent with this definition. In terms of human space exploration, >10 MeV particles pose a danger for astronauts outside of a vehicle during an EVA in space or on the lunar surface. Higher energy particles that can penetrate spacecraft shielding are a danger for astronauts throughout a mission, particularly exploration missions outside of Low-Earth Orbit (LEO). SRAG at NASA/JSC applies a threshold of >100 MeV proton flux exceeding 1 pfu to indicate when the particle environment is of concern. About half of the models provide predictions in that energy range, but many of those are physics-based models that face challenges in running in real time. Developing (and validating) empirical and machine learning models to make high energy proton predictions is also a challenge since SEP events that reach such energies are relatively rare. From a space radiation operations perspective, Table 11 demonstrates a need to expand forecasting to higher energies, particularly by “fast and light” models that can produce results ahead of an eruption or quickly following an eruptive event.

Table 10 shows that many of the models rely on flare or CME information to make their predictions, thus it is not surprising that most models in Table 11 provide post-eruption forecasts. The models that do make a prediction ahead of an eruption typically provide an All Clear and/or probability of occurrence. These models rely on information derived from magnetograms and take empirical or machine learning approaches. Indeed, pre-eruptive forecasting capabilities may be an area where machine learning algorithms can make advances over more traditional empirical models. This will remain to be seen until the application of ML to space weather forecasting matures.

Table 10 also indicates the current limited use of energetic electrons as model input and identifies an opportunity for future models to take advantage of their potential to provide an early warning of a proton event or as an effective discriminator to reduce false alarms. The detection of energetic electrons may be particularly important for space exploration missions as external electron detectors could be mounted on board the vehicle to create an early warning system at the spacecraft. Posner et al. (2020) have shown that electron measurements at Earth-L1 and Mars-L1 used with the RELEASE forecasting system could provide advanced warning of proton events throughout most of a mission to Mars.

Overall, Table 11 shows that the models, taken as a whole, provide good coverage of all listed quantities with high redundancy that could be used to address the above forecaster questions. However, a major current restriction is that most of the model outputs are not available to forecasters in real time. In some cases, this is due to the latency in observations, e.g., coronagraph images of CMEs. In other cases, models are not yet robust enough to run continuously or cannot produce output faster than real time. Space agencies around the world have developed forecasting dashboards for individual or a small number of SEP models, but the opportunity remains to make use of the diverse set of models that exist within the research community. Increased Research-to Operations (R2O) efforts are needed to prepare and onboard SEP models into operational settings.

### 4.3. Models running operationally or in real time

Significant effort is required to move a model from the development phase to running robustly in real time. Twelve of the SEP models discussed in this paper have undertaken these steps and are running in a real time or operational environment. Here, operational models are defined as those being served through the ESA Space Weather Service Network (https://swe.ssa.esa.int/) for its space weather users or utilized by the USAF or NOAA SWPC for space weather forecasting. These models have undergone a vetting process for robustness, stability, and performance by their respective operational agencies and have external users that monitor their forecasts for decision-making purposes. Five models satisfy these criteria: AFRL PPS, COMESEP<fnref n="23" />, PROTONS, RELEASE, and UMASEP.

An additional seven models are defined as running in real time: FORSPEF<fnref n="24" />, MAG4, SAWS-ASPECS<fnref n="25" />, SEP-MOD, SEPSTER, SEPSTER2D, and SPRINTS. These models have done the key work to ensure stability and are deployed on public-facing environments, like the applications developed at CCMC. Model forecasts are monitored by users on an experimental basis.

The ESA Space Weather Service Network provides a dashboard of forecasts to support a wide variety of end-users in order to mitigate the effects of space weather on their systems (Kruglanski et al., 2016). The Network is organized around five Expert Service Centres that each have an individual focus on Solar Weather, Heliospheric Weather, Space Radiation Environment, Ionospheric Weather and Geomagnetic Conditions.

The USAF Space Weather Operations Center (SpWOC) is the only Department of Defense (DoD) unit delivering 24/7 space weather information for the Total Force,

<footnote n="23">²³ SPARX predictions are used in COMESEP forecasting in the form of a pre-run database.</footnote>

<footnote n="24">²⁴ The PCA Model provides probabilistic forecasts within FORSPEF.</footnote>

<footnote n="25">²⁵ SOLPENCO2 predictions are used in SAWS-ASPECS forecasting in the form of a pre-run database.</footnote>
<figure><img src="image_19.png" /><figcaption>Fig. 19. The intensity SEP Scoreboard displaying forecasts for the 2017–09–10 SEP event. GOES proton measurements are shown as lines for >10 MeV (red), >50 MeV (blue), and >100 MeV (green). Forecasts from SEPSTER (stars), SEPSTER2D (triangles), SEPMOD (circles with dashed lines), and UMASEP (horizontal bars) for the same energies are in corresponding colors. HESPERIA/RELeASE forecasts are shown as black circles. The heat map on the left displays the observed GOES flux values and the peak flux predictions for >10 and >100 MeV from the various models. The SWPC Alerts bar at the top shows warnings, alerts, and event summaries issued by SWPC for X-ray fluxes, radio bursts, and protons.</figcaption></figure>

Defense, and Intelligence Community Agencies. They provide forecasts and tailored products at multiple classification levels through the Air Force Air Force Weather Enterprise (AFW-WEBS).

NOAA SWPC (https://www.swpc.noaa.gov) is a service center of the U.S. National Weather Service and provides official space weather forecasting services to industries in the United States and global users. SWPC's wide array of customers include the U.S. power grid, commercial airline industries, NASA, satellite operators, and others.

CCMC runs many of its models, including SEP models, continuously/real-time based on whether it is possible to do so and if the model is robust enough (https://ccmc.gsfc.nasa.gov/tools/continuous-run/). Real-time run outputs are primarily served via CCMC's integrated Space Weather Analysis (iSWA) web application which also displays space weather observations. All SEP models that are running in real-time at CCMC are available both via iSWA and the SEP Scoreboard (see Fig. 19). The SEP Scoreboard is one of many community-driven “Scoreboards” facilitated by the CCMC. Each scoreboard gathers, displays, and serves (via API) real-time forecasts in order to test the community’s predictive capabilities before event onset. The SEP Scoreboard project (https://ccmc.gsfc.nasa.gov/scoreboards/sep/) began in 2016 as an international effort and started receiving input from SRAG in 2018 in support of upcoming human space exploration missions. The SEP scoreboard captures SEP onset, duration, peak flux, probability, all-clear, and overall profile within three separate web applications: probability heat map and time-series, intensity heat-map and time-series, and all-clear.

Current forecasting methods participating in the SEP Scoreboard include: SAWS-ASPECS, MAG4, RELeASE, SEPMOD, SEPSTER, SEPSTER2D, SPRINTS, SWPC, and UMASEP. The GSU and iPATH models are currently being onboarded. The time series plots also show relevant SWPC alerts and MLSO K-Cor automatic CME detections or CME all-clear. The SEP Scoreboard is being monitored by SRAG operators as a test-case ahead of NASA's planned Artemis missions to the Moon. Table 12 lists each of the operational or real time models, the year that they were first implemented, the location of the model forecasts, and the end users. The model implementation history was collected through personal communication with the model developers and historical literature.

About 1/3 of the models summarized in this paper have performed the required work to transition their models to systems external to their home institutions where they have been configured to run continuously and robustly, regardless of data outages, corrupted observations, or other complicating factors. Significant effort is required on the part of model developers and the staff at the hosting institutions to onboard the models. This type of work does not typically fall within the scope of traditional funding opportunities, however it is necessary and important work for models that demonstrate benefit in the context of forecasting. Many model developers in the research community have expressed interest in working towards implementing their models in real time and are in the very early stages of this process. This worthwhile effort will require support from the various institutions that hold stake in space weather forecasting.
<figure><img src="image_20.png" /><figcaption>Fig. 19. The intensity SEP Scoreboard displaying forecasts for the 2017–09–10 SEP event. GOES proton measurements are shown as lines for >10 MeV (red), >50 MeV (blue), and >100 MeV (green). Forecasts from SEPSTER (stars), SEPSTER2D (triangles), SEPMOD (circles with dashed lines), and UMASEP (horizontal bars) for the same energies are in corresponding colors. HESPERIA/RELeASE forecasts are shown as black circles. The heat map on the left displays the observed GOES flux values and the peak flux predictions for >10 and >100 MeV from the various models. The SWPC Alerts bar at the top shows warnings, alerts, and event summaries issued by SWPC for X-ray fluxes, radio bursts, and protons.</figcaption></figure>

Defense, and Intelligence Community Agencies. They provide forecasts and tailored products at multiple classification levels through the Air Force Air Force Weather Enterprise (AFW-WEBS).

NOAA SWPC (https://www.swpc.noaa.gov) is a service center of the U.S. National Weather Service and provides official space weather forecasting services to industries in the United States and global users. SWPC's wide array of customers include the U.S. power grid, commercial airline industries, NASA, satellite operators, and others.

CCMC runs many of its models, including SEP models, continuously/real-time based on whether it is possible to do so and if the model is robust enough (https://ccmc.gsfc.nasa.gov/tools/continuous-run/). Real-time run outputs are primarily served via CCMC's integrated Space Weather Analysis (iSWA) web application which also displays space weather observations. All SEP models that are running in real-time at CCMC are available both via iSWA and the SEP Scoreboard (see Fig. 19). The SEP Scoreboard is one of many community-driven “Scoreboards” facilitated by the CCMC. Each scoreboard gathers, displays, and serves (via API) real-time forecasts in order to test the community’s predictive capabilities before event onset. The SEP Scoreboard project (https://ccmc.gsfc.nasa.gov/scoreboards/sep/) began in 2016 as an international effort and started receiving input from SRAG in 2018 in support of upcoming human space exploration missions. The SEP scoreboard captures SEP onset, duration, peak flux, probability, all-clear, and overall profile within three separate web applications: probability heat map and time-series, intensity heat-map and time-series, and all-clear.

Current forecasting methods participating in the SEP Scoreboard include: SAWS-ASPECS, MAG4, RELeASE, SEPMOD, SEPSTER, SEPSTER2D, SPRINTS, SWPC, and UMASEP. The GSU and iPATH models are currently being onboarded. The time series plots also show relevant SWPC alerts and MLSO K-Cor automatic CME detections or CME all-clear. The SEP Scoreboard is being monitored by SRAG operators as a test-case ahead of NASA's planned Artemis missions to the Moon. Table 12 lists each of the operational or real time models, the year that they were first implemented, the location of the model forecasts, and the end users. The model implementation history was collected through personal communication with the model developers and historical literature.

About 1/3 of the models summarized in this paper have performed the required work to transition their models to systems external to their home institutions where they have been configured to run continuously and robustly, regardless of data outages, corrupted observations, or other complicating factors. Significant effort is required on the part of model developers and the staff at the hosting institutions to onboard the models. This type of work does not typically fall within the scope of traditional funding opportunities, however it is necessary and important work for models that demonstrate benefit in the context of forecasting. Many model developers in the research community have expressed interest in working towards implementing their models in real time and are in the very early stages of this process. This worthwhile effort will require support from the various institutions that hold stake in space weather forecasting.
<table><caption>Table 12
SEP models that are running in operational and real time environments.</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Date of Implementation</th><th colspan="2">Operational Models</th></tr><tr><th>Location</th><th>End Users</th></tr></thead><tbody><tr><td>AFRL PPS</td><td>1987 (as early as 1976)</td><td>USAF Space Weather Analysis and Forecast System (SWAFS)</td><td>USAF Space Weather Operations Center (SpWOC)</td></tr><tr><td>COMESEP</td><td>2013 (real time),<br>2016 (operations)</td><td>ESA Space Weather Service Network</td><td>ESA space weather stakeholders, PECASUS global center for aviation users</td></tr><tr><td>PROTONS</td><td>1998 (earlier version running<br>from at least 1983)</td><td>NOAA SWPC</td><td>NOAA SWPC</td></tr><tr><td>HESPERIA<br>RELEASE</td><td>real time since 2008, operational<br>since 2016</td><td>ESA Space Weather Service Network,<br>HESPERIA, CCMC iSWA, SEP Scoreboard</td><td>ESA space weather stakeholders, NASA SRAG, public</td></tr><tr><td>UMASEP</td><td>real time since 2011, operational<br>since 2016</td><td>ESA Space Weather Service Network,<br>HESPERIA, University of Malaga, CCMC iSWA, SEP Scoreboard</td><td>ESA space weather stakeholders, NASA SRAG, public</td></tr></tbody></table>

<table><thead><tr><th rowspan="2">Model</th><th rowspan="2">Date of Implementation</th><th colspan="2">Real Time Models</th></tr><tr><th>Location</th><th>End Users</th></tr></thead><tbody><tr><td>FORSPEF</td><td>2015</td><td>NOA</td><td>registered users</td></tr><tr><td>MAG4</td><td>2019 (various real time locations<br>from 2011)</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SAWS-<br>ASPECS</td><td>2022</td><td>SEP Scoreboard, NOA</td><td>NASA SRAG, public</td></tr><tr><td>SEPMOD</td><td>2020</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SEPSTER</td><td>2020</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SEPSTER2D</td><td>2021</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SPRINTS</td><td>2022</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr></tbody></table>

## 4.4. Validation

Validation of SEP models has been non-uniform across the community. Validation efforts have ranged from subjective comparisons between model and data “by eye” for a few selected events, to thorough efforts that assess a variety of metrics and skill scores for a statistical sample of events. This non-uniformity is generally a consequence of the different computational needs and level of automation of each model. As the SEP modeling field matures, all SEP models should pursue a quantitative comparison to observations.

A number of steps can be taken to improve the quality, consistency, and ease of validation within the community. The creation of a standard set of SEP event definitions and the identification of community-accepted metrics would facilitate consistent validation and cross-model comparison. The curation of a shared data set of all observations required as input into the models (e.g., linked flare, CME, radio, and energetic electron measurements) and for comparison with model output (e.g., in situ particle data) would significantly reduce the workload for individual modelers and minimize duplicated efforts, and also remove disparities between model predictions that are due to the use of different input parameters. Lastly, a service with the ability to validate all types of SEP forecasts could be developed and made available to the community in a system such as CCMC’s Comprehensive Assessment of Models and Events using Library Tools (CAMEL) Framework (Rastätter et al., 2019).

Validation performed with the intent to demonstrate operational performance should aim to reproduce real-time operational scenarios as closely as possible, such as through participation in CCMC’s SEP Scoreboard<fnref n="26" />. Model forecasts should be issued without any tuning after the fact, and use only information available in real time as input into the model. It is also important to produce forecasts for time periods when no SEP event was observed, to assess false alarms and the validity of All Clear forecasts. Multi-point validation of 3D models for time periods when multiple spacecraft SEP observations are available at different locations can provide additional constraints and important insight into model performance. Lastly, forecasts should be output in energy ranges that match observational capabilities to ensure a meaningful comparison between model and observational data.

High-quality validation requires high-quality observational measurements to provide the “ground truth”, but the current spacecraft instruments that provide important SEP data sets have limitations that degrade their accuracy. In particular, the proton detectors on the operational GOES series of spacecraft have known problems with cross-talk between energy channels and contamination issues that impact the measurements, particularly during the onset of SEP events (Posner, 2007; Sandberg et al., 2014; Bruno, 2017; Rodriguez et al., 2017). The SOHO experiment has two science-quality detectors, EPHIN (Müller-Mellin et al., 1995), which only extends up to pro-

<footnote n="26">²⁶ https://ccmc.gscfc.nasa.gov/challenges/sep.php.</footnote>
<table><caption>Table 12
SEP models that are running in operational and real time environments.</caption><thead><tr><th rowspan="2">Model</th><th rowspan="2">Date of Implementation</th><th colspan="2">Operational Models</th></tr><tr><th>Location</th><th>End Users</th></tr></thead><tbody><tr><td>AFRL PPS</td><td>1987 (as early as 1976)</td><td>USAF Space Weather Analysis and Forecast System (SWAFS)</td><td>USAF Space Weather Operations Center (SpWOC)</td></tr><tr><td>COMESEP</td><td>2013 (real time),<br>2016 (operations)</td><td>ESA Space Weather Service Network</td><td>ESA space weather stakeholders, PECASUS global center for aviation users</td></tr><tr><td>PROTONS</td><td>1998 (earlier version running<br>from at least 1983)</td><td>NOAA SWPC</td><td>NOAA SWPC</td></tr><tr><td>HESPERIA<br>RELEASE</td><td>real time since 2008, operational<br>since 2016</td><td>ESA Space Weather Service Network,<br>HESPERIA, CCMC iSWA, SEP Scoreboard</td><td>ESA space weather stakeholders, NASA SRAG, public</td></tr><tr><td>UMASEP</td><td>real time since 2011, operational<br>since 2016</td><td>ESA Space Weather Service Network,<br>HESPERIA, University of Malaga, CCMC iSWA, SEP Scoreboard</td><td>ESA space weather stakeholders, NASA SRAG, public</td></tr></tbody></table>

<table><thead><tr><th rowspan="2">Model</th><th rowspan="2">Date of Implementation</th><th colspan="2">Real Time Models</th></tr><tr><th>Location</th><th>End Users</th></tr></thead><tbody><tr><td>FORSPEF</td><td>2015</td><td>NOA</td><td>registered users</td></tr><tr><td>MAG4</td><td>2019 (various real time locations<br>from 2011)</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SAWS-<br>ASPECS</td><td>2022</td><td>SEP Scoreboard, NOA</td><td>NASA SRAG, public</td></tr><tr><td>SEPMOD</td><td>2020</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SEPSTER</td><td>2020</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SEPSTER2D</td><td>2021</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr><tr><td>SPRINTS</td><td>2022</td><td>SEP Scoreboard</td><td>NASA SRAG, public</td></tr></tbody></table>

## 4.4. Validation

Validation of SEP models has been non-uniform across the community. Validation efforts have ranged from subjective comparisons between model and data “by eye” for a few selected events, to thorough efforts that assess a variety of metrics and skill scores for a statistical sample of events. This non-uniformity is generally a consequence of the different computational needs and level of automation of each model. As the SEP modeling field matures, all SEP models should pursue a quantitative comparison to observations.

A number of steps can be taken to improve the quality, consistency, and ease of validation within the community. The creation of a standard set of SEP event definitions and the identification of community-accepted metrics would facilitate consistent validation and cross-model comparison. The curation of a shared data set of all observations required as input into the models (e.g., linked flare, CME, radio, and energetic electron measurements) and for comparison with model output (e.g., in situ particle data) would significantly reduce the workload for individual modelers and minimize duplicated efforts, and also remove disparities between model predictions that are due to the use of different input parameters. Lastly, a service with the ability to validate all types of SEP forecasts could be developed and made available to the community in a system such as CCMC’s Comprehensive Assessment of Models and Events using Library Tools (CAMEL) Framework (Rastätter et al., 2019).

Validation performed with the intent to demonstrate operational performance should aim to reproduce real-time operational scenarios as closely as possible, such as through participation in CCMC’s SEP Scoreboard<fnref n="26" />. Model forecasts should be issued without any tuning after the fact, and use only information available in real time as input into the model. It is also important to produce forecasts for time periods when no SEP event was observed, to assess false alarms and the validity of All Clear forecasts. Multi-point validation of 3D models for time periods when multiple spacecraft SEP observations are available at different locations can provide additional constraints and important insight into model performance. Lastly, forecasts should be output in energy ranges that match observational capabilities to ensure a meaningful comparison between model and observational data.

High-quality validation requires high-quality observational measurements to provide the “ground truth”, but the current spacecraft instruments that provide important SEP data sets have limitations that degrade their accuracy. In particular, the proton detectors on the operational GOES series of spacecraft have known problems with cross-talk between energy channels and contamination issues that impact the measurements, particularly during the onset of SEP events (Posner, 2007; Sandberg et al., 2014; Bruno, 2017; Rodriguez et al., 2017). The SOHO experiment has two science-quality detectors, EPHIN (Müller-Mellin et al., 1995), which only extends up to pro-

<footnote n="26">²⁶ https://ccmc.gscfc.nasa.gov/challenges/sep.php.</footnote>
ton energies of 50 MeV, and ERNE (Torsti et al., 1995), which extends up to higher energies, but saturates during the highest intensity SEP events (Valtonen et al., 2009). These instruments (and also, for example, the High Energy Telescope(HET) s on the STEREO spacecraft) also have much lower backgrounds than the GOES proton detectors, and so can detect many more SEP events that do not meet the NOAA definition of an SEP event based on GOES data (e.g., Richardson et al., 2014), raising the question of how to define an “SEP event” for use in model validation. In the future, it would be valuable to fly operationally supported proton detectors near the Earth and at other locations (e.g., L4, see Section 4.1) that can overcome the limitations of the current instruments. For a detector to appropriately characterize SEP events, Vourlidas et al. (2021) specify that energy coverage for protons should extend from 1 to 1000 MeV. Kühl et al. (2020) describe such a detector that would build on the design of SOHO/EPHIN, but use lessons learned to envision a more capable detector across extended energy ranges and intensity levels. However, developing new SEP observatories and gathering event statistics from them will take considerable time. Therefore, near-term validation efforts should be aware of the limitations of the existing data archives, and further work should be done to correct the deficiencies to the greatest extent possible.

In conclusion, the SEP modeling community has developed a rich and diverse set of SEP models that exhibit a wide array of capabilities but currently have significant limitations, in particular arising from the gaps in real-time observations. If supported with the necessary observations, and with further developments, for example in computational capabilities and the application of artificial intelligence and machine learning, the field is poised for continued growth with a great potential to contribute significantly both to space weather operations and advances in the understanding of the physics of SEP events.

## Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgements

The ISEP project is supported by the Advanced Exploration Systems Division under the Human Exploration and Operations Mission Directorate of NASA and performed in support of the Human Health and Performance Contract for NASA (NNJ15HK11B).

The HESPERIA project was funded through the European Union's HORIZON 2020 research and Innovation Programme (Contract No 637324) and coordinated by the National Observatory of Athens in Greece.

PARADISE has received funding from the European Union's Horizon 2020 research and innovation programme

under grant agreement No 870405 (EUHFORIA 2.0) and the ESA project "Heliospheric modelling techniques" (Contract No. 4000133080/20/NL/CRS). The code was developed in the framework of the projects C14/19/089 (C1 project Internal Funds KU Leuven), G.0D07.19N (FWO-Vlaanderen), SIDC Data Exploitation (ESA Prodex-12), and Belspo project B2/191/P1/SWiM.

Predictive Science Inc. was supported by NASA grants 80NSSC19K0067 and 80NSSC20K0285, and NSF grant ICER1854790.

The development of the Sadykov et al. model was supported by NASA ESI grant 80NSSC20K0302 and NSF FDSS grant 1936361.

JGL and COL acknowledge support for the SEPMOD model development for CCMC under National Aeronautics and Space Administration (NASA) Grant/Cooperative Agreement 80NSSC20K1873 from GSFC to UCB/SSL, with additional support to COL from AFOSR Grant FA9550-16-1-0418.

SEPSTER was supported by the NASA Living With a Star Program (grants NNX15AB80G and NNG06EO90A), and by the CCMC/SRAG ISEP project.

SEPSTER2D was funded by the NASA/HSR program NNH19ZDA001NHSR, the Goddard Space Flight Center / Internal Scientist Funding Model (ISFM) grant HISFM18, and the Johnson Space Center / Space Radiation Analysis Group (SRAG) under the Integrated Solar Energetic Proton Alert/Warning System (ISEP) project.

SOLPENCO was funded by the ESA contract 14098/99/NL/MM and its validation by the Spanish Ministerio de Educación y Ciencia under the project AYA2004-03022.

SOLPENCO2 was developed under ESA's SEPEM project (Crosby et al., 2015) and updated during ESA's SOL2UP project (Aran et al., 2017), both projects under ESA's contract n. 20162/06/NL/JD and 4000114116/15/NL/HK, respectively.

SPARX was initially developed as part of the European Union-funded Seventh Framework Programme (FP7) COMESEP project.

The development of the SPREADFAST framework has been funded by a contract to the ESA.

SD and CW acknowledge support from NERC via grant NE/V002864/1.

## Appendix A. Acronyms

An extensive list of acronyms used throughout this paper and their meanings have been compiled here for clarity and organized into multiple tables. General acronyms including physics terminology, instruments, and institutions are listed in Table A.13. Acronyms related to Machine Learning are described in Table A.14 and those related to validation are in Table A.15. Acronyms for the SEP models reviewed in this paper are found in Table A.16 while those related to other space weather models are found in Table A.17.
ton energies of 50 MeV, and ERNE (Torsti et al., 1995), which extends up to higher energies, but saturates during the highest intensity SEP events (Valtonen et al., 2009). These instruments (and also, for example, the High Energy Telescope(HET) s on the STEREO spacecraft) also have much lower backgrounds than the GOES proton detectors, and so can detect many more SEP events that do not meet the NOAA definition of an SEP event based on GOES data (e.g., Richardson et al., 2014), raising the question of how to define an “SEP event” for use in model validation. In the future, it would be valuable to fly operationally supported proton detectors near the Earth and at other locations (e.g., L4, see Section 4.1) that can overcome the limitations of the current instruments. For a detector to appropriately characterize SEP events, Vourlidas et al. (2021) specify that energy coverage for protons should extend from 1 to 1000 MeV. Kühl et al. (2020) describe such a detector that would build on the design of SOHO/EPHIN, but use lessons learned to envision a more capable detector across extended energy ranges and intensity levels. However, developing new SEP observatories and gathering event statistics from them will take considerable time. Therefore, near-term validation efforts should be aware of the limitations of the existing data archives, and further work should be done to correct the deficiencies to the greatest extent possible.

In conclusion, the SEP modeling community has developed a rich and diverse set of SEP models that exhibit a wide array of capabilities but currently have significant limitations, in particular arising from the gaps in real-time observations. If supported with the necessary observations, and with further developments, for example in computational capabilities and the application of artificial intelligence and machine learning, the field is poised for continued growth with a great potential to contribute significantly both to space weather operations and advances in the understanding of the physics of SEP events.

## Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Acknowledgements

The ISEP project is supported by the Advanced Exploration Systems Division under the Human Exploration and Operations Mission Directorate of NASA and performed in support of the Human Health and Performance Contract for NASA (NNJ15HK11B).

The HESPERIA project was funded through the European Union's HORIZON 2020 research and Innovation Programme (Contract No 637324) and coordinated by the National Observatory of Athens in Greece.

PARADISE has received funding from the European Union's Horizon 2020 research and innovation programme

under grant agreement No 870405 (EUHFORIA 2.0) and the ESA project "Heliospheric modelling techniques" (Contract No. 4000133080/20/NL/CRS). The code was developed in the framework of the projects C14/19/089 (C1 project Internal Funds KU Leuven), G.0D07.19N (FWO-Vlaanderen), SIDC Data Exploitation (ESA Prodex-12), and Belspo project B2/191/P1/SWiM.

Predictive Science Inc. was supported by NASA grants 80NSSC19K0067 and 80NSSC20K0285, and NSF grant ICER1854790.

The development of the Sadykov et al. model was supported by NASA ESI grant 80NSSC20K0302 and NSF FDSS grant 1936361.

JGL and COL acknowledge support for the SEPMOD model development for CCMC under National Aeronautics and Space Administration (NASA) Grant/Cooperative Agreement 80NSSC20K1873 from GSFC to UCB/SSL, with additional support to COL from AFOSR Grant FA9550-16-1-0418.

SEPSTER was supported by the NASA Living With a Star Program (grants NNX15AB80G and NNG06EO90A), and by the CCMC/SRAG ISEP project.

SEPSTER2D was funded by the NASA/HSR program NNH19ZDA001NHSR, the Goddard Space Flight Center / Internal Scientist Funding Model (ISFM) grant HISFM18, and the Johnson Space Center / Space Radiation Analysis Group (SRAG) under the Integrated Solar Energetic Proton Alert/Warning System (ISEP) project.

SOLPENCO was funded by the ESA contract 14098/99/NL/MM and its validation by the Spanish Ministerio de Educación y Ciencia under the project AYA2004-03022.

SOLPENCO2 was developed under ESA's SEPEM project (Crosby et al., 2015) and updated during ESA's SOL2UP project (Aran et al., 2017), both projects under ESA's contract n. 20162/06/NL/JD and 4000114116/15/NL/HK, respectively.

SPARX was initially developed as part of the European Union-funded Seventh Framework Programme (FP7) COMESEP project.

The development of the SPREADFAST framework has been funded by a contract to the ESA.

SD and CW acknowledge support from NERC via grant NE/V002864/1.

## Appendix A. Acronyms

An extensive list of acronyms used throughout this paper and their meanings have been compiled here for clarity and organized into multiple tables. General acronyms including physics terminology, instruments, and institutions are listed in Table A.13. Acronyms related to Machine Learning are described in Table A.14 and those related to validation are in Table A.15. Acronyms for the SEP models reviewed in this paper are found in Table A.16 while those related to other space weather models are found in Table A.17.
<table><caption>Table A.13
General acronyms.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>ACE</td><td>Advanced Composition Explorer</td></tr><tr><td>ACE/EPAM</td><td>Electron, Proton, and Alpha Monitor</td></tr><tr><td>AFRL</td><td>Air Force Research Laboratory</td></tr><tr><td>API</td><td>Application Programming Interface</td></tr><tr><td>AR</td><td>Active Region</td></tr><tr><td>ASTRON</td><td>Netherlands Institute for Radio Astronomy</td></tr><tr><td>AU</td><td>Astronomical Unit</td></tr><tr><td>AW</td><td>Angular Width</td></tr><tr><td>BIRA-IASB</td><td>Royal Belgian Institute for Space Aeronomy</td></tr><tr><td>CACTUS</td><td>Computer Aided CME Tracking catalog</td></tr><tr><td>CAMEL</td><td>Comprehensive Assessment of Models and Events using Library Tools</td></tr><tr><td>CBF</td><td>Coronal Bright Front</td></tr><tr><td>CCMC</td><td>Community Coordinated Modeling Center</td></tr><tr><td>CCOR</td><td>Compact Coronagraph</td></tr><tr><td>CDAW</td><td>Coordinated Data Analysis Web</td></tr><tr><td>CIR</td><td>Corotating Interaction Region</td></tr><tr><td>CIRES</td><td>Cooperative Institute for Research in Environmental Sciences</td></tr><tr><td>CME</td><td>Coronal Mass Ejection</td></tr><tr><td>Cobpoint</td><td>Connecting-with-Observer point</td></tr><tr><td>DONKI</td><td>Space Weather Database of Notifications, Knowledge, Information</td></tr><tr><td>DSA</td><td>Diffusive Shock Acceleration</td></tr><tr><td>DSCOVR</td><td>Deep Space Climate Observatory</td></tr><tr><td>ESA</td><td>European Space Agency</td></tr><tr><td>ESP</td><td>Energetic Storm Particles</td></tr><tr><td>EU</td><td>European Union</td></tr><tr><td>EUV</td><td>Extreme Ultraviolet</td></tr><tr><td>FTE</td><td>Focused Transport Equation</td></tr><tr><td>FQ</td><td>Flare-Quiet</td></tr><tr><td>GLE</td><td>Ground Level Event</td></tr><tr><td>GOES</td><td>Geostationary Operational Environment Satellite</td></tr><tr><td>GOES/EPS</td><td>Energetic Particle Sensor</td></tr><tr><td>GOES/EPEAD</td><td>Energetic Proton Electron and Alpha Detector</td></tr><tr><td>GOES/HEPAD</td><td>High Energy Proton and Alpha Detector</td></tr><tr><td>GOES/XRS</td><td>X-ray Sensor</td></tr><tr><td>GONG</td><td>Global Oscillation Network Group</td></tr><tr><td>GSFC</td><td>NASA Goddard Space Flight Center</td></tr><tr><td>GUI</td><td>Graphical User Interface</td></tr><tr><td>HARP</td><td>HMI Active Region Patch</td></tr><tr><td>HEK</td><td>Heliospheric Event Knowledgebase</td></tr><tr><td>HESPERIA</td><td>High Energy Solar Particle Events forecasting and Analysis</td></tr><tr><td>Hinode/EIS</td><td>EUV Imaging Spectrometer</td></tr><tr><td>IA-BIS</td><td>Institute of Astronomy, Bulgarian Academy of Sciences</td></tr><tr><td>ICME</td><td>Interplanetary Coronal Mass Ejection</td></tr><tr><td>IDL</td><td>Interactive Data Language</td></tr><tr><td>IEAP</td><td>Institute of Experimental and Applied Physics</td></tr><tr><td>IMF</td><td>Interplanetary Magnetic Field</td></tr><tr><td>IMP</td><td>Interplanetary Monitoring Platform (series of satellites)</td></tr><tr><td>INAF</td><td>National Institute for Astrophysics</td></tr><tr><td>IP</td><td>Interplanetary</td></tr><tr><td>IRAP</td><td>Institut de Recherche en Astrophysique et Planetologie</td></tr><tr><td>ISEP</td><td>Integrated Solar Energetic Proton Event Alert/Warning System</td></tr><tr><td>iSWA</td><td>integrated Space Weather Analysis System</td></tr><tr><td>ISWAT</td><td>International Space Weather Action Teams</td></tr><tr><td>IZMIRAN</td><td>Pushkov Institute of Terrestrial Magnetism, Ionosphere and Radio Wave Propagation</td></tr><tr><td>JSC</td><td>NASA Johnson Space Center</td></tr><tr><td>JSOC</td><td>Stanford Joint Science Operations Center</td></tr><tr><td>KU</td><td>Katholieke Universiteit</td></tr><tr><td>L1, L4</td><td>Langrangian point 1, 4</td></tr><tr><td>LEO</td><td>Low Earth Orbit</td></tr><tr><td>LMSAL</td><td>Lockheed Martin Solar and Astrophysics Laboratory</td></tr><tr><td>LOS</td><td>Line of Sight</td></tr><tr><td>MDI</td><td>Michelson Doppler Imager</td></tr></tbody></table>

(continued on next page)
<table><caption>Table A.13
General acronyms.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>ACE</td><td>Advanced Composition Explorer</td></tr><tr><td>ACE/EPAM</td><td>Electron, Proton, and Alpha Monitor</td></tr><tr><td>AFRL</td><td>Air Force Research Laboratory</td></tr><tr><td>API</td><td>Application Programming Interface</td></tr><tr><td>AR</td><td>Active Region</td></tr><tr><td>ASTRON</td><td>Netherlands Institute for Radio Astronomy</td></tr><tr><td>AU</td><td>Astronomical Unit</td></tr><tr><td>AW</td><td>Angular Width</td></tr><tr><td>BIRA-IASB</td><td>Royal Belgian Institute for Space Aeronomy</td></tr><tr><td>CACTUS</td><td>Computer Aided CME Tracking catalog</td></tr><tr><td>CAMEL</td><td>Comprehensive Assessment of Models and Events using Library Tools</td></tr><tr><td>CBF</td><td>Coronal Bright Front</td></tr><tr><td>CCMC</td><td>Community Coordinated Modeling Center</td></tr><tr><td>CCOR</td><td>Compact Coronagraph</td></tr><tr><td>CDAW</td><td>Coordinated Data Analysis Web</td></tr><tr><td>CIR</td><td>Corotating Interaction Region</td></tr><tr><td>CIRES</td><td>Cooperative Institute for Research in Environmental Sciences</td></tr><tr><td>CME</td><td>Coronal Mass Ejection</td></tr><tr><td>Cobpoint</td><td>Connecting-with-Observer point</td></tr><tr><td>DONKI</td><td>Space Weather Database of Notifications, Knowledge, Information</td></tr><tr><td>DSA</td><td>Diffusive Shock Acceleration</td></tr><tr><td>DSCOVR</td><td>Deep Space Climate Observatory</td></tr><tr><td>ESA</td><td>European Space Agency</td></tr><tr><td>ESP</td><td>Energetic Storm Particles</td></tr><tr><td>EU</td><td>European Union</td></tr><tr><td>EUV</td><td>Extreme Ultraviolet</td></tr><tr><td>FTE</td><td>Focused Transport Equation</td></tr><tr><td>FQ</td><td>Flare-Quiet</td></tr><tr><td>GLE</td><td>Ground Level Event</td></tr><tr><td>GOES</td><td>Geostationary Operational Environment Satellite</td></tr><tr><td>GOES/EPS</td><td>Energetic Particle Sensor</td></tr><tr><td>GOES/EPEAD</td><td>Energetic Proton Electron and Alpha Detector</td></tr><tr><td>GOES/HEPAD</td><td>High Energy Proton and Alpha Detector</td></tr><tr><td>GOES/XRS</td><td>X-ray Sensor</td></tr><tr><td>GONG</td><td>Global Oscillation Network Group</td></tr><tr><td>GSFC</td><td>NASA Goddard Space Flight Center</td></tr><tr><td>GUI</td><td>Graphical User Interface</td></tr><tr><td>HARP</td><td>HMI Active Region Patch</td></tr><tr><td>HEK</td><td>Heliospheric Event Knowledgebase</td></tr><tr><td>HESPERIA</td><td>High Energy Solar Particle Events forecasting and Analysis</td></tr><tr><td>Hinode/EIS</td><td>EUV Imaging Spectrometer</td></tr><tr><td>IA-BIS</td><td>Institute of Astronomy, Bulgarian Academy of Sciences</td></tr><tr><td>ICME</td><td>Interplanetary Coronal Mass Ejection</td></tr><tr><td>IDL</td><td>Interactive Data Language</td></tr><tr><td>IEAP</td><td>Institute of Experimental and Applied Physics</td></tr><tr><td>IMF</td><td>Interplanetary Magnetic Field</td></tr><tr><td>IMP</td><td>Interplanetary Monitoring Platform (series of satellites)</td></tr><tr><td>INAF</td><td>National Institute for Astrophysics</td></tr><tr><td>IP</td><td>Interplanetary</td></tr><tr><td>IRAP</td><td>Institut de Recherche en Astrophysique et Planetologie</td></tr><tr><td>ISEP</td><td>Integrated Solar Energetic Proton Event Alert/Warning System</td></tr><tr><td>iSWA</td><td>integrated Space Weather Analysis System</td></tr><tr><td>ISWAT</td><td>International Space Weather Action Teams</td></tr><tr><td>IZMIRAN</td><td>Pushkov Institute of Terrestrial Magnetism, Ionosphere and Radio Wave Propagation</td></tr><tr><td>JSC</td><td>NASA Johnson Space Center</td></tr><tr><td>JSOC</td><td>Stanford Joint Science Operations Center</td></tr><tr><td>KU</td><td>Katholieke Universiteit</td></tr><tr><td>L1, L4</td><td>Langrangian point 1, 4</td></tr><tr><td>LEO</td><td>Low Earth Orbit</td></tr><tr><td>LMSAL</td><td>Lockheed Martin Solar and Astrophysics Laboratory</td></tr><tr><td>LOS</td><td>Line of Sight</td></tr><tr><td>MDI</td><td>Michelson Doppler Imager</td></tr></tbody></table>

(continued on next page)
<table><caption>Table A.13 (continued)</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>MHD</td><td>Magnetohydrodynamic</td></tr><tr><td>NASA</td><td>National Aeronautics and Space Administration</td></tr><tr><td>NKUA</td><td>National and Kapodistrian University of Athens</td></tr><tr><td>NLGC</td><td>Nonlinear Guiding Center theory</td></tr><tr><td>NM</td><td>Neutron Monitor</td></tr><tr><td>NOA</td><td>National Observatory of Athens</td></tr><tr><td>NOAA</td><td>National Oceanics and Atmospheric Administration</td></tr><tr><td>NRT</td><td>Near Real Time</td></tr><tr><td>NSO</td><td>National Solar Observatory</td></tr><tr><td>PAMELA</td><td>Payload for Antimatter Matter Exploration and Light-nuclei Astrophysics (satellite experiment)</td></tr><tr><td>PC</td><td>Principal Components</td></tr><tr><td>PCA</td><td>Principal Component Analysis</td></tr><tr><td>pfu</td><td>Particle Flux Unit</td></tr><tr><td>PIL</td><td>Polarity Inversion Line</td></tr><tr><td>PSI</td><td>Predictive Science Inc</td></tr><tr><td>PSP</td><td>Parker Solar Probe</td></tr><tr><td>QLT</td><td>Quasi-linear Theory</td></tr><tr><td>R2O</td><td>Research to Operations</td></tr><tr><td>RSTN</td><td>Radio Solar Telescope Network</td></tr><tr><td>SC</td><td>Solar Cycle</td></tr><tr><td>SDE</td><td>Stochastic Differential Equation</td></tr><tr><td>SDO</td><td>Solar Dynamics Observatory</td></tr><tr><td>SDO/AIA</td><td>Advanced Imaging Assembly</td></tr><tr><td>SDO/HMI</td><td>Helioseismic Magnetic Imager</td></tr><tr><td>SECCHI</td><td>Sun Earth Connection Coronal and Heliospheric Investigation</td></tr><tr><td>SEON</td><td>Solar Electro-Optical Observatory Network</td></tr><tr><td>SEP</td><td>Solar Energetic Particles</td></tr><tr><td>SEPEM</td><td>ESA's Solar Energetic Particle Environment Modelling</td></tr><tr><td>SEPEM RDS</td><td>SEPEM Reference Data Set</td></tr><tr><td>SF</td><td>Solar Flare</td></tr><tr><td>sfu</td><td>Solar flux unit</td></tr><tr><td>SHARP</td><td>Space-weather HMI Active Region Patch</td></tr><tr><td>SHINE</td><td>Solar Heliospheric and INterplanetary Environment workshop</td></tr><tr><td>SMARP</td><td>Space-Weather MDI Active Region Patches</td></tr><tr><td>SO</td><td>Solar Orbiter</td></tr><tr><td>SOHO</td><td>Solar and Heliospheric Observatory</td></tr><tr><td>SOHO/COSTEP</td><td>Comprehensive Suprathermal and Energetic Particle Analyzer</td></tr><tr><td>SOHO/EPHIN</td><td>Electron Proton and Helium Instrument</td></tr><tr><td>SOHO/EIT</td><td>Extreme-Ultraviolet Imaging Telescope</td></tr><tr><td>SOHO/ERNE</td><td>Energetic and Relativistic Nuclei and Electron</td></tr><tr><td>SOHO/LASCO</td><td>Large Angle and Spectrometric Coronagraph Experiment</td></tr><tr><td>SOHO/MDI</td><td>Michelson Doppler Imager</td></tr><tr><td>Solar DEMON</td><td>Solar Dimming and EUV wave Monitor</td></tr><tr><td>SOLSTICE</td><td>Solar Storms and Terrestrial Impacts Center</td></tr><tr><td>SOON</td><td>Solar Observing Optical Network</td></tr><tr><td>SPARC</td><td>Space Applications and Research Consultancy</td></tr><tr><td>SPE</td><td>Solar Particle Event, Solar Proton Event</td></tr><tr><td>SRAG</td><td>Space Radiation Analysis Group</td></tr><tr><td>SRS</td><td>Solar Region Summary</td></tr><tr><td>STEREO</td><td>Solar Terrestrial Relations Observatory (A &amp; B)</td></tr><tr><td>STEREO/LET</td><td>Low-Energy Telescope</td></tr><tr><td>STEREO/HET</td><td>High-Energy Telescope</td></tr><tr><td>SWFO-L1</td><td>Space Weather Follow On - L1</td></tr><tr><td>SWPC</td><td>Space Weather Prediction Center</td></tr><tr><td>SWRI</td><td>Southwest Research Institute</td></tr><tr><td>SXR</td><td>Soft X-rays</td></tr><tr><td>USAF</td><td>United States Air Force</td></tr><tr><td>UTU</td><td>University of Turku</td></tr><tr><td>VSWMC</td><td>Virtual Space Weather Modelling Centre</td></tr><tr><td>WAVES</td><td>Radio and Plasma Wave Experiment</td></tr></tbody></table>
<table><caption>Table A.13 (continued)</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>MHD</td><td>Magnetohydrodynamic</td></tr><tr><td>NASA</td><td>National Aeronautics and Space Administration</td></tr><tr><td>NKUA</td><td>National and Kapodistrian University of Athens</td></tr><tr><td>NLGC</td><td>Nonlinear Guiding Center theory</td></tr><tr><td>NM</td><td>Neutron Monitor</td></tr><tr><td>NOA</td><td>National Observatory of Athens</td></tr><tr><td>NOAA</td><td>National Oceanics and Atmospheric Administration</td></tr><tr><td>NRT</td><td>Near Real Time</td></tr><tr><td>NSO</td><td>National Solar Observatory</td></tr><tr><td>PAMELA</td><td>Payload for Antimatter Matter Exploration and Light-nuclei Astrophysics (satellite experiment)</td></tr><tr><td>PC</td><td>Principal Components</td></tr><tr><td>PCA</td><td>Principal Component Analysis</td></tr><tr><td>pfu</td><td>Particle Flux Unit</td></tr><tr><td>PIL</td><td>Polarity Inversion Line</td></tr><tr><td>PSI</td><td>Predictive Science Inc</td></tr><tr><td>PSP</td><td>Parker Solar Probe</td></tr><tr><td>QLT</td><td>Quasi-linear Theory</td></tr><tr><td>R2O</td><td>Research to Operations</td></tr><tr><td>RSTN</td><td>Radio Solar Telescope Network</td></tr><tr><td>SC</td><td>Solar Cycle</td></tr><tr><td>SDE</td><td>Stochastic Differential Equation</td></tr><tr><td>SDO</td><td>Solar Dynamics Observatory</td></tr><tr><td>SDO/AIA</td><td>Advanced Imaging Assembly</td></tr><tr><td>SDO/HMI</td><td>Helioseismic Magnetic Imager</td></tr><tr><td>SECCHI</td><td>Sun Earth Connection Coronal and Heliospheric Investigation</td></tr><tr><td>SEON</td><td>Solar Electro-Optical Observatory Network</td></tr><tr><td>SEP</td><td>Solar Energetic Particles</td></tr><tr><td>SEPEM</td><td>ESA's Solar Energetic Particle Environment Modelling</td></tr><tr><td>SEPEM RDS</td><td>SEPEM Reference Data Set</td></tr><tr><td>SF</td><td>Solar Flare</td></tr><tr><td>sfu</td><td>Solar flux unit</td></tr><tr><td>SHARP</td><td>Space-weather HMI Active Region Patch</td></tr><tr><td>SHINE</td><td>Solar Heliospheric and INterplanetary Environment workshop</td></tr><tr><td>SMARP</td><td>Space-Weather MDI Active Region Patches</td></tr><tr><td>SO</td><td>Solar Orbiter</td></tr><tr><td>SOHO</td><td>Solar and Heliospheric Observatory</td></tr><tr><td>SOHO/COSTEP</td><td>Comprehensive Suprathermal and Energetic Particle Analyzer</td></tr><tr><td>SOHO/EPHIN</td><td>Electron Proton and Helium Instrument</td></tr><tr><td>SOHO/EIT</td><td>Extreme-Ultraviolet Imaging Telescope</td></tr><tr><td>SOHO/ERNE</td><td>Energetic and Relativistic Nuclei and Electron</td></tr><tr><td>SOHO/LASCO</td><td>Large Angle and Spectrometric Coronagraph Experiment</td></tr><tr><td>SOHO/MDI</td><td>Michelson Doppler Imager</td></tr><tr><td>Solar DEMON</td><td>Solar Dimming and EUV wave Monitor</td></tr><tr><td>SOLSTICE</td><td>Solar Storms and Terrestrial Impacts Center</td></tr><tr><td>SOON</td><td>Solar Observing Optical Network</td></tr><tr><td>SPARC</td><td>Space Applications and Research Consultancy</td></tr><tr><td>SPE</td><td>Solar Particle Event, Solar Proton Event</td></tr><tr><td>SRAG</td><td>Space Radiation Analysis Group</td></tr><tr><td>SRS</td><td>Solar Region Summary</td></tr><tr><td>STEREO</td><td>Solar Terrestrial Relations Observatory (A &amp; B)</td></tr><tr><td>STEREO/LET</td><td>Low-Energy Telescope</td></tr><tr><td>STEREO/HET</td><td>High-Energy Telescope</td></tr><tr><td>SWFO-L1</td><td>Space Weather Follow On - L1</td></tr><tr><td>SWPC</td><td>Space Weather Prediction Center</td></tr><tr><td>SWRI</td><td>Southwest Research Institute</td></tr><tr><td>SXR</td><td>Soft X-rays</td></tr><tr><td>USAF</td><td>United States Air Force</td></tr><tr><td>UTU</td><td>University of Turku</td></tr><tr><td>VSWMC</td><td>Virtual Space Weather Modelling Centre</td></tr><tr><td>WAVES</td><td>Radio and Plasma Wave Experiment</td></tr></tbody></table>
<table><caption>Table A.14
Acronyms related to machine learning.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>AI</td><td>Artificial Intelligence</td></tr><tr><td>CART</td><td>Classification And Regression Trees</td></tr><tr><td>CNN</td><td>Convolutional Neural Network</td></tr><tr><td>DT</td><td>Decision Trees</td></tr><tr><td>ET</td><td>Extremely Randomized Trees</td></tr><tr><td>LR</td><td>Logistic Regression</td></tr><tr><td>ML</td><td>Machine Learning</td></tr><tr><td>MLE</td><td>Mean Log Error</td></tr><tr><td>MLP</td><td>Multi-layer Perceptron</td></tr><tr><td>MV</td><td>Missing Values</td></tr><tr><td>MVTS</td><td>multivariate time series</td></tr><tr><td>NN</td><td>Neural Network</td></tr><tr><td>RF</td><td>Random Forest</td></tr><tr><td>RBF</td><td>Radial Basis Function</td></tr><tr><td>SVM</td><td>Support Vector Machines</td></tr><tr><td>TSC</td><td>Time Series Classifier</td></tr><tr><td>VAR</td><td>Vector Autoregression model</td></tr><tr><td>XGB</td><td>Extreme Gradient Boosting</td></tr></tbody></table>

<table><caption>Table A.15
Acronyms related to validation.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>AUC</td><td>Area Under the Curve</td></tr><tr><td>AWT</td><td>Advanced Warning Time</td></tr><tr><td>AWT</td><td>Average Warning Time</td></tr><tr><td>BS</td><td>Brier Score</td></tr><tr><td>CDF</td><td>Cumulative Distribution Function or Cumulative Probability Function</td></tr><tr><td>CSI</td><td>Critical Success Index</td></tr><tr><td>FAR</td><td>False Alarm Rate</td></tr><tr><td>FN</td><td>False Negative</td></tr><tr><td>FP</td><td>False Positive</td></tr><tr><td>HSS</td><td>Heidke Skill Score</td></tr><tr><td>OA</td><td>Overall Accuracy</td></tr><tr><td>PC</td><td>Percent Correct</td></tr><tr><td>pt or PT</td><td>Probability Threshold</td></tr><tr><td>POD</td><td>probability of Detection</td></tr><tr><td>POFD</td><td>Probability of False Detection</td></tr><tr><td>ROC</td><td>Receiver Operator Characteristic</td></tr><tr><td>TN</td><td>True Negative</td></tr><tr><td>TP</td><td>True Positive</td></tr><tr><td>TSS</td><td>True Skill Statistic</td></tr><tr><td>WTSS</td><td>Weighted True Skill Statistic</td></tr></tbody></table>

<table><caption>Table A.16
Acronyms for the SEP models reviewed in this paper.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>ADEPT</td><td>Air Force Dynamic Energetic Particle Tool</td></tr><tr><td>AFRL PPS</td><td>Air Force Research Laboratory Proton Prediction System</td></tr><tr><td>AMPS</td><td>Adaptive Mesh Particle Simulator</td></tr><tr><td>COMESEP</td><td>Coronal Mass Ejections and Solar Energetic Particles</td></tr><tr><td>EPREM</td><td>Energetic Particle Radiation Environment Module</td></tr><tr><td>ESPERTA</td><td>Empirical model for Solar Proton Event Real Time Alert</td></tr><tr><td>FORSPEF</td><td>Forecasting Solar Particle Events and Flares</td></tr><tr><td>iPATH</td><td>improved Particle Acceleration and Transport in the Heliosphere</td></tr><tr><td>MAG4</td><td>Magnetogram Forecast</td></tr><tr><td>MagPy</td><td>Magnetogram Forecast in Python</td></tr><tr><td>MEMPSEP</td><td>Multivariate Ensemble of Models for Probabilistic SEP prediction</td></tr><tr><td>M-FLAMPA</td><td>Multiple-Field-Line-Advection Model for Particle Acceleration</td></tr><tr><td>PARADISE</td><td>Particle Radiation Asset Directed at Interplanetary Space Exploration</td></tr><tr><td>PCA</td><td>Principal Component Analysis model</td></tr><tr><td>RELEASE</td><td>Relativistic Electron Alert System for Exploration</td></tr><tr><td>SAWS-ASPECS</td><td>SEP Advanced Warning System, Advanced Solar Particle Events Casting System</td></tr><tr><td>SEPMOD</td><td>Solar Energetic Particle Model</td></tr><tr><td>SEPSTER</td><td>SEP prediction derived from STEREO observations</td></tr><tr><td>SOLPENCO</td><td>SOLar Particle Engineering Code</td></tr><tr><td>SPARX</td><td>Solar Particle Radiation swX</td></tr><tr><td>SPREAdFAST</td><td>Solar Particle Radiation Environment Analysis and Forecasting – Acceleration and Scattering Transport</td></tr><tr><td>SPRINTS</td><td>Space Radiation Intelligence System</td></tr><tr><td>STAT</td><td>Solar Particle Event (SPE) Threat Assessment Tool</td></tr><tr><td>UMASEP</td><td>University of Malaga Solar Energetic Particles</td></tr></tbody></table>
<table><caption>Table A.14
Acronyms related to machine learning.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>AI</td><td>Artificial Intelligence</td></tr><tr><td>CART</td><td>Classification And Regression Trees</td></tr><tr><td>CNN</td><td>Convolutional Neural Network</td></tr><tr><td>DT</td><td>Decision Trees</td></tr><tr><td>ET</td><td>Extremely Randomized Trees</td></tr><tr><td>LR</td><td>Logistic Regression</td></tr><tr><td>ML</td><td>Machine Learning</td></tr><tr><td>MLE</td><td>Mean Log Error</td></tr><tr><td>MLP</td><td>Multi-layer Perceptron</td></tr><tr><td>MV</td><td>Missing Values</td></tr><tr><td>MVTS</td><td>multivariate time series</td></tr><tr><td>NN</td><td>Neural Network</td></tr><tr><td>RF</td><td>Random Forest</td></tr><tr><td>RBF</td><td>Radial Basis Function</td></tr><tr><td>SVM</td><td>Support Vector Machines</td></tr><tr><td>TSC</td><td>Time Series Classifier</td></tr><tr><td>VAR</td><td>Vector Autoregression model</td></tr><tr><td>XGB</td><td>Extreme Gradient Boosting</td></tr></tbody></table>

<table><caption>Table A.15
Acronyms related to validation.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>AUC</td><td>Area Under the Curve</td></tr><tr><td>AWT</td><td>Advanced Warning Time</td></tr><tr><td>AWT</td><td>Average Warning Time</td></tr><tr><td>BS</td><td>Brier Score</td></tr><tr><td>CDF</td><td>Cumulative Distribution Function or Cumulative Probability Function</td></tr><tr><td>CSI</td><td>Critical Success Index</td></tr><tr><td>FAR</td><td>False Alarm Rate</td></tr><tr><td>FN</td><td>False Negative</td></tr><tr><td>FP</td><td>False Positive</td></tr><tr><td>HSS</td><td>Heidke Skill Score</td></tr><tr><td>OA</td><td>Overall Accuracy</td></tr><tr><td>PC</td><td>Percent Correct</td></tr><tr><td>pt or PT</td><td>Probability Threshold</td></tr><tr><td>POD</td><td>probability of Detection</td></tr><tr><td>POFD</td><td>Probability of False Detection</td></tr><tr><td>ROC</td><td>Receiver Operator Characteristic</td></tr><tr><td>TN</td><td>True Negative</td></tr><tr><td>TP</td><td>True Positive</td></tr><tr><td>TSS</td><td>True Skill Statistic</td></tr><tr><td>WTSS</td><td>Weighted True Skill Statistic</td></tr></tbody></table>

<table><caption>Table A.16
Acronyms for the SEP models reviewed in this paper.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>ADEPT</td><td>Air Force Dynamic Energetic Particle Tool</td></tr><tr><td>AFRL PPS</td><td>Air Force Research Laboratory Proton Prediction System</td></tr><tr><td>AMPS</td><td>Adaptive Mesh Particle Simulator</td></tr><tr><td>COMESEP</td><td>Coronal Mass Ejections and Solar Energetic Particles</td></tr><tr><td>EPREM</td><td>Energetic Particle Radiation Environment Module</td></tr><tr><td>ESPERTA</td><td>Empirical model for Solar Proton Event Real Time Alert</td></tr><tr><td>FORSPEF</td><td>Forecasting Solar Particle Events and Flares</td></tr><tr><td>iPATH</td><td>improved Particle Acceleration and Transport in the Heliosphere</td></tr><tr><td>MAG4</td><td>Magnetogram Forecast</td></tr><tr><td>MagPy</td><td>Magnetogram Forecast in Python</td></tr><tr><td>MEMPSEP</td><td>Multivariate Ensemble of Models for Probabilistic SEP prediction</td></tr><tr><td>M-FLAMPA</td><td>Multiple-Field-Line-Advection Model for Particle Acceleration</td></tr><tr><td>PARADISE</td><td>Particle Radiation Asset Directed at Interplanetary Space Exploration</td></tr><tr><td>PCA</td><td>Principal Component Analysis model</td></tr><tr><td>REleASE</td><td>Relativistic Electron Alert System for Exploration</td></tr><tr><td>SAWS-ASPECS</td><td>SEP Advanced Warning System, Advanced Solar Particle Events Casting System</td></tr><tr><td>SEPMOD</td><td>Solar Energetic Particle Model</td></tr><tr><td>SEPSTER</td><td>SEP prediction derived from STEREO observations</td></tr><tr><td>SOLPENCO</td><td>SOLar Particle Engineering Code</td></tr><tr><td>SPARX</td><td>Solar Particle Radiation swX</td></tr><tr><td>SPREAdFAST</td><td>Solar Particle Radiation Environment Analysis and Forecasting – Acceleration and Scattering Transport</td></tr><tr><td>SPRINTS</td><td>Space Radiation Intelligence System</td></tr><tr><td>STAT</td><td>Solar Particle Event (SPE) Threat Assessment Tool</td></tr><tr><td>UMASEP</td><td>University of Malaga Solar Energetic Particles</td></tr></tbody></table>
<table><caption>Table A.17
Acronyms related to space weather models.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>ADAPT</td><td>Air Force Data Assimilative Photospheric flux Transport</td></tr><tr><td>AWSoM</td><td>Alfven-Wave driven sOlar wind Model</td></tr><tr><td>CASHeW</td><td>Coronal Analysis of Shocks and Waves</td></tr><tr><td>CORHEL</td><td>Corona-Heliosphere model</td></tr><tr><td>EEGGL</td><td>Eruptive Event Generator Gibson-Low</td></tr><tr><td>EMMREM</td><td>Earth-Moon-Mars Radiation Environment Module</td></tr><tr><td>EUHFORIA</td><td>European Heliospheric Forecasting Information Asset</td></tr><tr><td>FLARECAST</td><td>Flare Likelihood And Region Eruption foreCASTing</td></tr><tr><td>GL</td><td>Gibson-Low</td></tr><tr><td>MAS</td><td>Magnetohydrodynamic Algorithm outside a Sphere</td></tr><tr><td>NLGC</td><td>Nonlinear Guiding Center Theory</td></tr><tr><td>PFSS</td><td>Potential Field Source Surface</td></tr><tr><td>PREDICCS</td><td>Predictions of Radiation from REleASE, EMMREM, and Data Incorporating the CRaTER, COSTEP, and other SEP measurements</td></tr><tr><td>PROSPER</td><td>Probabilistic Solar Particle Event forecasting</td></tr><tr><td>RBSL</td><td>Regularized Biot-Savart Laws</td></tr><tr><td>REL</td><td>Reference Event List</td></tr><tr><td>SaP</td><td>Shock and Particle</td></tr><tr><td>SOLPACS</td><td>SOLar Particle Acceleration in Coronal Shocks</td></tr><tr><td>S2M</td><td>Synthetic Shock Model</td></tr><tr><td>SWMF</td><td>Space Weather Modeling Framework</td></tr><tr><td>TDM</td><td>Titov-Démoulin</td></tr><tr><td>UMASEP/PC</td><td>Poorly Connected module</td></tr><tr><td>UMASEP/WC</td><td>Well Connected module</td></tr><tr><td>UMASOD</td><td>University of Malaga predictor from Solar Data</td></tr><tr><td>WSA</td><td>Wang-Sheeley-Arge model</td></tr></tbody></table>

## References

Afanasiev, A., Battarbee, M., Vainio, R., 2015. Self-consistent Monte Carlo simulations of proton acceleration in coronal shocks: Effect of anisotropic pitch-angle scattering of particles. Astron. Astroph. 584, A81.

Ahmadzadeh, A., Aydin, B., Georgoulis, M.K., Kempton, D.J., Mahajan, S.S., Angryk, R.A., 2021. How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events. Astrophys. J. Suppl 254 (2), 23.

Alberti, T., Laurenza, M., Cliver, E.W., 2019. Forecasting solar proton events by using the ESPERTA model. Nuovo Cimento C 42 (1), 40.

Alberti, T., Laurenza, M., Cliver, E.W., Storini, M., Consolini, G., Lepreti, F., 2017. Solar Activity from 2006 to 2014 and Short-term Forecasts of Solar Proton Events Using the ESPERTA Model. Astrophys. J. 838 (1), 59.

Alissandrakis, C.E., 1981. On the computation of constant alpha force-free magnetic field. Astron. Astroph. 100 (1), 197–200.

Aminalragia-Giamini, S., Raptis, S., Anastasiadis, A., Tsigkanos, A., Sandberg, I., Papaioannou, A., Papadimitriou, C., Jiggens, P., Aran, A., Daglis, I.A., 2021. Solar Energetic Particle Event occurrence prediction using Solar Flare Soft X-ray measurements and Machine Learning. J. Space Weather Space Climate 11, 59.

Anastasiadis, A., Papaioannou, A., Sandberg, I., Georgoulis, M., Tziotziou, K., Kouloumvakos, A., Jiggens, P., 2017. Predicting Flares and Solar Energetic Particle Events: The FORSPEF Tool. Sol. Phys. 292 (9), 134.

Angryk, R.A., Martens, P.C., Aydin, B., Kempton, D., Mahajan, S.S., Basodi, S., Ahmadzadeh, A., Cai, X., Filali Boubrahimi, S., Hamdi, S. M., Schuh, M.A., Georgoulis, M.K., 2020. Multivariate time series dataset for space weather data analytics. Sci. Data 7 (1), 227.

Aran, A., 2007. Synthesis of proton flux profiles of SEP events associated with interplanetary shocks. The tool SOLPENCO. Ph.D. thesis, Universitat de Barcelona, Barcelona, Spain.

Aran, A., Lario, D., Sanahuja, B., Marsden, R.G., Dryer, M., Fry, C.D., McKenna-Lawlor, S.M.P., 2007. Modeling and forecasting solar energetic particle events at Mars: the event on 6 March 1989. Astron. Astroph. 469 (3), 1123–1134.

Aran, A., Pacheco, D., Agueda, N., Sanahuja, B., 2017. Updating SOLPENCO2 and New Analysis on Downstream FLuence (SOL2UP) Project. Final Report, ESA/ESTEC Contract 4000114116/15/NL/HK, 1–90.

Aran, A., Rodriguez-Gasen, R., Sanahuja, B., Jabos, C., Poedts, S., 2011. Wp410: Initial and boundary conditions for the shock-and-particle model. SEPEM Tehnical Report, ESA/ESTEC Contract 20162/06/NL/JD, 1–58.

Aran, A., Sanahuja, B., Lario, D., 2005a. A first step towards proton flux forecasting. Adv. Space Res. 36 (12), 2333–2338.

Aran, A., Sanahuja, B., Lario, D., 2005b. Fluxes and fluences of SEP events derived from SOLPENCO. Ann. Geophys. 23 (9), 3047–3053.

Aran, A., Sanahuja, B., Lario, D., 2006. SOLPENCO: A solar particle engineering code. Adv. Space Res. 37 (6), 1240–1246.

Aran, A., Sanahuja, B., Lario, D., 2008. Comparing proton fluxes of central meridian SEP events with those predicted by SOLPENCO. Adv. Space Res 42 (9), 1492–1499.

Arge, C.N., Pizzo, V.J., 2000. Improvement in the prediction of solar wind conditions using near-real time solar magnetic field updates. J. Geophys. Res.: Space Phys. 105 (A5), 10465–10479, URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/1999JA000262.

Aschwanden, M.J., Freeland, S.L., 2012. Automated Solar Flare Statistics in Soft X-Rays over 37 Years of GOES Observations: The Invariance of Self-organized Criticality during Three Solar Cycles. Astrophys. J. 754 (2), 112.

Axford, W.I., Leer, E., Skadron, G., Jan. 1977. The Acceleration of Cosmic Rays by Shock Waves. In: International Cosmic Ray Conference. Vol. 11 of International Cosmic Ray Conference. p. 132.

Bain, H.M., Mays, M.L., Luhmann, J.G., Li, Y., Jian, L.K., Odstrcil, D., 2016. Shock Connectivity in the August 2010 and July 2012 Solar
<table><caption>Table A.17
Acronyms related to space weather models.</caption><thead><tr><td>Acronym</td><td>Meaning</td></tr></thead><tbody><tr><td>ADAPT</td><td>Air Force Data Assimilative Photospheric flux Transport</td></tr><tr><td>AWSoM</td><td>Alfven-Wave driven sOlar wind Model</td></tr><tr><td>CASHeW</td><td>Coronal Analysis of Shocks and Waves</td></tr><tr><td>CORHEL</td><td>Corona-Heliosphere model</td></tr><tr><td>EEGGL</td><td>Eruptive Event Generator Gibson-Low</td></tr><tr><td>EMMREM</td><td>Earth-Moon-Mars Radiation Environment Module</td></tr><tr><td>EUHFORIA</td><td>European Heliospheric Forecasting Information Asset</td></tr><tr><td>FLARECAST</td><td>Flare Likelihood And Region Eruption foreCASTing</td></tr><tr><td>GL</td><td>Gibson-Low</td></tr><tr><td>MAS</td><td>Magnetohydrodynamic Algorithm outside a Sphere</td></tr><tr><td>NLGC</td><td>Nonlinear Guiding Center Theory</td></tr><tr><td>PFSS</td><td>Potential Field Source Surface</td></tr><tr><td>PREDICCS</td><td>Predictions of Radiation from REleASE, EMMREM, and Data Incorporating the CRaTER, COSTEP, and other SEP measurements</td></tr><tr><td>PROSPER</td><td>Probabilistic Solar Particle Event forecasting</td></tr><tr><td>RBSL</td><td>Regularized Biot-Savart Laws</td></tr><tr><td>REL</td><td>Reference Event List</td></tr><tr><td>SaP</td><td>Shock and Particle</td></tr><tr><td>SOLPACS</td><td>SOLar Particle Acceleration in Coronal Shocks</td></tr><tr><td>S2M</td><td>Synthetic Shock Model</td></tr><tr><td>SWMF</td><td>Space Weather Modeling Framework</td></tr><tr><td>TDM</td><td>Titov-Démoulin</td></tr><tr><td>UMASEP/PC</td><td>Poorly Connected module</td></tr><tr><td>UMASEP/WC</td><td>Well Connected module</td></tr><tr><td>UMASOD</td><td>University of Malaga predictor from Solar Data</td></tr><tr><td>WSA</td><td>Wang-Sheeley-Arge model</td></tr></tbody></table>

## References

Afanasiev, A., Battarbee, M., Vainio, R., 2015. Self-consistent Monte Carlo simulations of proton acceleration in coronal shocks: Effect of anisotropic pitch-angle scattering of particles. Astron. Astroph. 584, A81.

Ahmadzadeh, A., Aydin, B., Georgoulis, M.K., Kempton, D.J., Mahajan, S.S., Angryk, R.A., 2021. How to Train Your Flare Prediction Model: Revisiting Robust Sampling of Rare Events. Astrophys. J. Suppl 254 (2), 23.

Alberti, T., Laurenza, M., Cliver, E.W., 2019. Forecasting solar proton events by using the ESPERTA model. Nuovo Cimento C 42 (1), 40.

Alberti, T., Laurenza, M., Cliver, E.W., Storini, M., Consolini, G., Lepreti, F., 2017. Solar Activity from 2006 to 2014 and Short-term Forecasts of Solar Proton Events Using the ESPERTA Model. Astrophys. J. 838 (1), 59.

Alissandrakis, C.E., 1981. On the computation of constant alpha force-free magnetic field. Astron. Astroph. 100 (1), 197–200.

Aminalragia-Giamini, S., Raptis, S., Anastasiadis, A., Tsigkanos, A., Sandberg, I., Papaioannou, A., Papadimitriou, C., Jiggens, P., Aran, A., Daglis, I.A., 2021. Solar Energetic Particle Event occurrence prediction using Solar Flare Soft X-ray measurements and Machine Learning. J. Space Weather Space Climate 11, 59.

Anastasiadis, A., Papaioannou, A., Sandberg, I., Georgoulis, M., Tziotziou, K., Kouloumvakos, A., Jiggens, P., 2017. Predicting Flares and Solar Energetic Particle Events: The FORSPEF Tool. Sol. Phys. 292 (9), 134.

Angryk, R.A., Martens, P.C., Aydin, B., Kempton, D., Mahajan, S.S., Basodi, S., Ahmadzadeh, A., Cai, X., Filali Boubrahimi, S., Hamdi, S. M., Schuh, M.A., Georgoulis, M.K., 2020. Multivariate time series dataset for space weather data analytics. Sci. Data 7 (1), 227.

Aran, A., 2007. Synthesis of proton flux profiles of SEP events associated with interplanetary shocks. The tool SOLPENCO. Ph.D. thesis, Universitat de Barcelona, Barcelona, Spain.

Aran, A., Lario, D., Sanahuja, B., Marsden, R.G., Dryer, M., Fry, C.D., McKenna-Lawlor, S.M.P., 2007. Modeling and forecasting solar energetic particle events at Mars: the event on 6 March 1989. Astron. Astroph. 469 (3), 1123–1134.

Aran, A., Pacheco, D., Agueda, N., Sanahuja, B., 2017. Updating SOLPENCO2 and New Analysis on Downstream FLuence (SOL2UP) Project. Final Report, ESA/ESTEC Contract 4000114116/15/NL/HK, 1–90.

Aran, A., Rodriguez-Gasen, R., Sanahuja, B., Jabos, C., Poedts, S., 2011. Wp410: Initial and boundary conditions for the shock-and-particle model. SEPEM Tehnical Report, ESA/ESTEC Contract 20162/06/NL/JD, 1–58.

Aran, A., Sanahuja, B., Lario, D., 2005a. A first step towards proton flux forecasting. Adv. Space Res. 36 (12), 2333–2338.

Aran, A., Sanahuja, B., Lario, D., 2005b. Fluxes and fluences of SEP events derived from SOLPENCO. Ann. Geophys. 23 (9), 3047–3053.

Aran, A., Sanahuja, B., Lario, D., 2006. SOLPENCO: A solar particle engineering code. Adv. Space Res. 37 (6), 1240–1246.

Aran, A., Sanahuja, B., Lario, D., 2008. Comparing proton fluxes of central meridian SEP events with those predicted by SOLPENCO. Adv. Space Res 42 (9), 1492–1499.

Arge, C.N., Pizzo, V.J., 2000. Improvement in the prediction of solar wind conditions using near-real time solar magnetic field updates. J. Geophys. Res.: Space Phys. 105 (A5), 10465–10479, URL https://agupubs.onlinelibrary.wiley.com/doi/abs/10.1029/1999JA000262.

Aschwanden, M.J., Freeland, S.L., 2012. Automated Solar Flare Statistics in Soft X-Rays over 37 Years of GOES Observations: The Invariance of Self-organized Criticality during Three Solar Cycles. Astrophys. J. 754 (2), 112.

Axford, W.I., Leer, E., Skadron, G., Jan. 1977. The Acceleration of Cosmic Rays by Shock Waves. In: International Cosmic Ray Conference. Vol. 11 of International Cosmic Ray Conference. p. 132.

Bain, H.M., Mays, M.L., Luhmann, J.G., Li, Y., Jian, L.K., Odstrcil, D., 2016. Shock Connectivity in the August 2010 and July 2012 Solar
Energetic Particle Events Inferred from Observations and ENLIL Modeling. Astrophys. J. 825 (1), 1.

Bain, H.M., Steenburgh, R.A., Onsager, T.G., Stitely, E.M., 2021. A Summary of National Oceanic and Atmospheric Administration Space Weather Prediction Center Proton Event Forecast Performance and Skill. Space Weather 19 (7), e2020SW002670.

Balch, C.C., 1999. SEC proton prediction model: verification and analysis. Radiat. Meas. 30 (3), 231–250.

Balch, C.C., 2008. Updated verification of the Space Weather Prediction Center's solar energetic particle prediction model. Space Weather 6 (1), S01001.

Barouch, E., Gros, M., Masse, P., 1971. The Solar Longitude Dependence of Proton Event Delay Time. Sol. Phys. 19 (2), 483–493.

Battarbee, M., Vainio, R., Laitinen, T., Hietala, H., 2013. Injection of thermal and suprathelial seed particles into coronal shocks of varying obliquity. Astron. Astroph. 558, A110.

Bobra, M.G., Sun, X., Hoeksema, J.T., Turmon, M., Liu, Y., Hayashi, K., Barnes, G., Leka, K.D., 2014. The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: SHARPs - Space-Weather HMI Active Region Patches. Sol. Phys. 289 (9), 3549–3578.

Bobra, M.G., Wright, P.J., Sun, X., Turmon, M.J., 2021. SMARPs and SHARPs: Two Solar Cycles of Active Region Data. Astrophys. J. Suppl. 256 (2), 26.

Borovikov, D., Sokolov, I.V., Manchester, W.B., Jin, M., Gombosi, T.I., 2017. Eruptive event generator based on the Gibson-Low magnetic configuration. J. Geophys. Res.: Space Phys. 122 (8), 7979–7984.

Borovikov, D., Sokolov, I.V., Roussev, I.I., Taktakishvili, A., Gombosi, T.I., 2018. Toward a Quantitative Model for Simulation and Forecast of Solar Energetic Particle Production during Gradual Events. I. Magnetohydrodynamic Background Coupled to the SEP Model. Astrophys. J. 864 (1), 88.

Borovikov, D., Sokolov, I.V., Tóth, G., 2015. An efficient second-order accurate and continuous interpolation for block-adaptive grids. J. Comput. Phys. 297, 599–610.

Boubrahimi, S.F., Aydin, B., Martens, P., Angryk, R., 2017. On the prediction of >100 mev solar energetic particle events using goes satellite data. In: 2017 IEEE International Conference on Big Data (Big Data). pp. 2533–2542.

Bruno, A., 2017. Calibration of the GOES 13/15 high-energy proton detectors based on the PAMELA solar energetic particle observations. Space Weather 15 (9), 1191–1202.

Bruno, A., Bazilevskaya, G.A., Boezio, M., Christian, E.R., de Nolfo, G. A., Martucci, M., Merge', M., Mikhailov, V.V., Munini, R., Richardson, I.G., Ryan, J.M., 2018. Solar Energetic Particle Events Observed by the PAMELA Mission. Astrophys. J. 862 (2), 97.

Bruno, A., Richardson, I.G., 2021. Empirical Model of 10–130 MeV Solar Energetic Particle Spectra at 1 AU Based on Coronal Mass Ejection Speed and Direction. Sol. Phys. 296 (2), 36.

Byrne, J.P., Long, D.M., Gallagher, P.T., Bloomfield, D.S., Maloney, S. A., McAteer, R.T.J., Morgan, H., Habbal, S.R., 2013. Improved methods for determining the kinematics of coronal mass ejections and coronal waves. Astron. Astroph. 557, A96.

Cane, H.V., Erickson, W.C., Prestage, N.P., 2002. Solar flares, type III radio bursts, coronal mass ejections, and energetic particles. J. Geophys. Res. 107 (A10), 1315.

Cane, H.V., Lario, D., 2006. An Introduction to CMEs and Energetic Particles. Space Sci. Rev. 123 (1–3), 45–56.

Cane, H.V., Reames, D.V., von Rosenvinge, T.T., 1988. The role of interplanetary shocks in the longitude distribution of solar energetic particles. J. Geophys. Res. 93 (A9), 9555–9567.

Cane, H.V., Richardson, I.G., von Rosenvinge, T.T., 2010. A study of solar energetic particle events of 1997–2006: Their composition and associations. J. Geophys. Res. 115 (A8), A08101.

Chen, J.H., Schwadron, N.A., Möbius, E., Gorby, M., 2015. Modeling interstellar pickup ion distributions in corotating interaction regions inside 1 AU. J. Geophys. Res.: Space Phys. 120 (11), 9269–9280.

Cliver, E.W., Kahler, S.W., Reames, D.V., 2004. Coronal Shocks and Solar Energetic Proton Events. Astrophys. J. 605 (2), 902–910.

Cliver, E.W., Mekhaldi, F., Muscheler, R., 2020. Solar Longitude Distribution of High-energy Proton Flares: Fluences and Spectra. Astrophys. J. Lett. 900 (1), L11.

Cortes, C., Vapnik, V., 1995. Support-vector networks. Mach. Learn. 20 (3), 273–297.

Crosby, N., Heynderickx, D., Jiggens, P., Aran, A., Sanahuja, B., Truscott, P., Lei, F., Jacobs, C., Poedts, S., Gabriel, S., Sandberg, I., Glover, A., Hilgers, A., 2015. SEPEM: A tool for statistical modeling the solar energetic particle environment. Space Weather 13 (7), 406–426.

Crosby, N.B., Veronig, A., Robbrecht, E., Vrsnak, B., Vennerstrom, S., Malandraki, O., Dalla, S., Rodriguez, L., Srivastava, N., Hesse, M., Odstrcil, D., Comesep Consortium, Nov. 2012. Forecasting the space weather impact: The COMESEP project. In: Hu, Q., Li, G., Zank, G. P., Ao, X., Verkhoglyadova, O., Adams, J.H. (Eds.), Space Weather: the Space Radiation Environment: 11th Annual International Astrophysics Conference. Vol. 1500 of American Institute of Physics Conference Series. pp. 159–164.

Dalla, S., Marsh, M.S., Kelly, J., Laitinen, T., 2013. Solar energetic particle drifts in the Parker spiral. J. Geophys. Res.: Space Phys. 118 (10), 5979–5985.

Dalla, S., Swalwell, B., Battarbee, M., Marsh, M.S., Laitinen, T., Proctor, S.J., Aug. 2018. Application of Test Particle Simulations to Solar Energetic Particle Forecasting. In: Foullon, C., Malandraki, O.E. (Eds.), Space Weather of the Heliosphere: Processes and Forecasts. vol. 335. pp. 268–271.

Davis, J., Goadrich, M., 2006. The Relationship between Precision-Recall and ROC Curves. In: Cohen, W.W., Moore, A. (Eds.), Proceedings of the 23rd International Conference on Machine Learning. ICML '06. Association for Computing Machinery, New York, NY, USA, p. 233–240.

Dayeh, M.A., Desai, M.I., Dwyer, J.R., Rassoul, H.K., Mason, G.M., Mazur, J.E., 2009. Composition and Spectral Properties of the 1 AU Quiet-Time Suprathelial Ion Population During Solar Cycle 23. Astrophys. J. 693 (2), 1588–1600.

Dayeh, M.A., Desai, M.I., Kozarev, K., Schwadron, N.A., Townsend, L. W., PourArsalan, M., Zeitlin, C., Hatcher, R.B., 2010. Modeling proton intensity gradients and radiation dose equivalents in the inner heliosphere using EMMREM: May 2003 solar events. Space Weather 8 (9), S00E07.

Dayeh, M.A., Desai, M.I., Mason, G.M., Ebert, R.W., Farahat, A., 2017. Origin and Properties of Quiet-time 0.11-1.28 MeV Nucleon-1 Heavy-ion Population near 1 au. Astrophys. J. 835 (2), 155.

Deng, H., Runger, G., Tuv, E., Vladimir, M., 2013. A time series forest for classification and feature extraction. Inf. Sci. 239, 142–153.

Desai, M., Giacalone, J., 2016. Large gradual solar energetic particle events. Living Rev. Sol. Phys. 13 (1), 3.

Desai, M. I., Mitchell, D. G., Szalay, J. R., Roelof, E. C., Giacalone, J., Hill, M. E., McComas, D. J., Christian, E. R., Schwadron, N. A., McNutt, R. L., J., Wiedenbeck, M. E., Joyce, C., Cohen, C. M. S., Ebert, R. W., Dayeh, M. A., Allen, R. C., Davis, A. J., Krimigis, S. M., Leske, R. A., Matthaeus, W. H., Malandraki, O., Mewaldt, R. A., Labrador, A., Stone, E. C., Bale, S. D., Pulupa, M., MacDowall, R. J., Kasper, J. C., Feb. 2020. Properties of Suprathelial-through-energetic He Ions Associated with Stream Interaction Regions Observed over the Parker Solar Probe's First Two Orbits. Astrophys. J. Suppl. 246 (2), 56.

Dierckxsens, M., Tziotziou, K., Dalla, S., Patsou, I., Marsh, M.S., Crosby, N.B., Malandraki, O., Tsiropoula, G., 2015. Relationship between Solar Energetic Particles and Properties of Flares and CMEs: Statistical Analysis of Solar Cycle 23 Events. Sol. Phys. 290 (3), 841–874.

Ding, Z.-Y., Li, G., Hu, J.-X., Fu, S., 2020. Modeling the 2017 September 10 solar energetic particle event using the iPATH model. Res. Astron. Astrophys. 20 (9), 145.

Downs, C., Lionello, R., Mikić, Z., Linker, J.A., Velli, M., Dec. 2016. Closed-field Coronal Heating Driven by Wave Turbulence. Astrophys. J. 832 (2), 180.
Energetic Particle Events Inferred from Observations and ENLIL Modeling. Astrophys. J. 825 (1), 1.

Bain, H.M., Steenburgh, R.A., Onsager, T.G., Stitely, E.M., 2021. A Summary of National Oceanic and Atmospheric Administration Space Weather Prediction Center Proton Event Forecast Performance and Skill. Space Weather 19 (7), e2020SW002670.

Balch, C.C., 1999. SEC proton prediction model: verification and analysis. Radiat. Meas. 30 (3), 231–250.

Balch, C.C., 2008. Updated verification of the Space Weather Prediction Center's solar energetic particle prediction model. Space Weather 6 (1), S01001.

Barouch, E., Gros, M., Masse, P., 1971. The Solar Longitude Dependence of Proton Event Delay Time. Sol. Phys. 19 (2), 483–493.

Battarbee, M., Vainio, R., Laitinen, T., Hietala, H., 2013. Injection of thermal and suprathermal seed particles into coronal shocks of varying obliquity. Astron. Astroph. 558, A110.

Bobra, M.G., Sun, X., Hoeksema, J.T., Turmon, M., Liu, Y., Hayashi, K., Barnes, G., Leka, K.D., 2014. The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: SHARPs - Space-Weather HMI Active Region Patches. Sol. Phys. 289 (9), 3549–3578.

Bobra, M.G., Wright, P.J., Sun, X., Turmon, M.J., 2021. SMARPs and SHARPs: Two Solar Cycles of Active Region Data. Astrophys. J. Suppl. 256 (2), 26.

Borovikov, D., Sokolov, I.V., Manchester, W.B., Jin, M., Gombosi, T.I., 2017. Eruptive event generator based on the Gibson-Low magnetic configuration. J. Geophys. Res.: Space Phys. 122 (8), 7979–7984.

Borovikov, D., Sokolov, I.V., Roussev, I.I., Taktakishvili, A., Gombosi, T.I., 2018. Toward a Quantitative Model for Simulation and Forecast of Solar Energetic Particle Production during Gradual Events. I. Magnetohydrodynamic Background Coupled to the SEP Model. Astrophys. J. 864 (1), 88.

Borovikov, D., Sokolov, I.V., Tóth, G., 2015. An efficient second-order accurate and continuous interpolation for block-adaptive grids. J. Comput. Phys. 297, 599–610.

Boubrahimi, S.F., Aydin, B., Martens, P., Angryk, R., 2017. On the prediction of >100 mev solar energetic particle events using goes satellite data. In: 2017 IEEE International Conference on Big Data (Big Data). pp. 2533–2542.

Bruno, A., 2017. Calibration of the GOES 13/15 high-energy proton detectors based on the PAMELA solar energetic particle observations. Space Weather 15 (9), 1191–1202.

Bruno, A., Bazilevskaya, G.A., Boezio, M., Christian, E.R., de Nolfo, G. A., Martucci, M., Merge', M., Mikhailov, V.V., Munini, R., Richardson, I.G., Ryan, J.M., 2018. Solar Energetic Particle Events Observed by the PAMELA Mission. Astrophys. J. 862 (2), 97.

Bruno, A., Richardson, I.G., 2021. Empirical Model of 10–130 MeV Solar Energetic Particle Spectra at 1 AU Based on Coronal Mass Ejection Speed and Direction. Sol. Phys. 296 (2), 36.

Byrne, J.P., Long, D.M., Gallagher, P.T., Bloomfield, D.S., Maloney, S. A., McAteer, R.T.J., Morgan, H., Habbal, S.R., 2013. Improved methods for determining the kinematics of coronal mass ejections and coronal waves. Astron. Astroph. 557, A96.

Cane, H.V., Erickson, W.C., Prestage, N.P., 2002. Solar flares, type III radio bursts, coronal mass ejections, and energetic particles. J. Geophys. Res. 107 (A10), 1315.

Cane, H.V., Lario, D., 2006. An Introduction to CMEs and Energetic Particles. Space Sci. Rev. 123 (1–3), 45–56.

Cane, H.V., Reames, D.V., von Rosenvinge, T.T., 1988. The role of interplanetary shocks in the longitude distribution of solar energetic particles. J. Geophys. Res. 93 (A9), 9555–9567.

Cane, H.V., Richardson, I.G., von Rosenvinge, T.T., 2010. A study of solar energetic particle events of 1997–2006: Their composition and associations. J. Geophys. Res. 115 (A8), A08101.

Chen, J.H., Schwadron, N.A., Möbius, E., Gorby, M., 2015. Modeling interstellar pickup ion distributions in corotating interaction regions inside 1 AU. J. Geophys. Res.: Space Phys. 120 (11), 9269–9280.

Cliver, E.W., Kahler, S.W., Reames, D.V., 2004. Coronal Shocks and Solar Energetic Proton Events. Astrophys. J. 605 (2), 902–910.

Cliver, E.W., Mekhaldi, F., Muscheler, R., 2020. Solar Longitude Distribution of High-energy Proton Flares: Fluences and Spectra. Astrophys. J. Lett. 900 (1), L11.

Cortes, C., Vapnik, V., 1995. Support-vector networks. Mach. Learn. 20 (3), 273–297.

Crosby, N., Heynderickx, D., Jiggens, P., Aran, A., Sanahuja, B., Truscott, P., Lei, F., Jacobs, C., Poedts, S., Gabriel, S., Sandberg, I., Glover, A., Hilgers, A., 2015. SEPEM: A tool for statistical modeling the solar energetic particle environment. Space Weather 13 (7), 406–426.

Crosby, N.B., Veronig, A., Robbrecht, E., Vrsnak, B., Vennerstrom, S., Malandraki, O., Dalla, S., Rodriguez, L., Srivastava, N., Hesse, M., Odstrcil, D., Comesep Consortium, Nov. 2012. Forecasting the space weather impact: The COMESEP project. In: Hu, Q., Li, G., Zank, G. P., Ao, X., Verkhoglyadova, O., Adams, J.H. (Eds.), Space Weather: the Space Radiation Environment: 11th Annual International Astrophysics Conference. Vol. 1500 of American Institute of Physics Conference Series. pp. 159–164.

Dalla, S., Marsh, M.S., Kelly, J., Laitinen, T., 2013. Solar energetic particle drifts in the Parker spiral. J. Geophys. Res.: Space Phys. 118 (10), 5979–5985.

Dalla, S., Swalwell, B., Battarbee, M., Marsh, M.S., Laitinen, T., Proctor, S.J., Aug. 2018. Application of Test Particle Simulations to Solar Energetic Particle Forecasting. In: Foullon, C., Malandraki, O.E. (Eds.), Space Weather of the Heliosphere: Processes and Forecasts. vol. 335. pp. 268–271.

Davis, J., Goadrich, M., 2006. The Relationship between Precision-Recall and ROC Curves. In: Cohen, W.W., Moore, A. (Eds.), Proceedings of the 23rd International Conference on Machine Learning. ICML '06. Association for Computing Machinery, New York, NY, USA, p. 233–240.

Dayeh, M.A., Desai, M.I., Dwyer, J.R., Rassoul, H.K., Mason, G.M., Mazur, J.E., 2009. Composition and Spectral Properties of the 1 AU Quiet-Time Suprathedral Ion Population During Solar Cycle 23. Astrophys. J. 693 (2), 1588–1600.

Dayeh, M.A., Desai, M.I., Kozarev, K., Schwadron, N.A., Townsend, L. W., PourArsalan, M., Zeitlin, C., Hatcher, R.B., 2010. Modeling proton intensity gradients and radiation dose equivalents in the inner heliosphere using EMMREM: May 2003 solar events. Space Weather 8 (9), S00E07.

Dayeh, M.A., Desai, M.I., Mason, G.M., Ebert, R.W., Farahat, A., 2017. Origin and Properties of Quiet-time 0.11-1.28 MeV Nucleon-1 Heavy-ion Population near 1 au. Astrophys. J. 835 (2), 155.

Deng, H., Runger, G., Tuv, E., Vladimir, M., 2013. A time series forest for classification and feature extraction. Inf. Sci. 239, 142–153.

Desai, M., Giacalone, J., 2016. Large gradual solar energetic particle events. Living Rev. Sol. Phys. 13 (1), 3.

Desai, M. I., Mitchell, D. G., Szalay, J. R., Roelof, E. C., Giacalone, J., Hill, M. E., McComas, D. J., Christian, E. R., Schwadron, N. A., McNutt, R. L., J., Wiedenbeck, M. E., Joyce, C., Cohen, C. M. S., Ebert, R. W., Dayeh, M. A., Allen, R. C., Davis, A. J., Krimigis, S. M., Leske, R. A., Matthaeus, W. H., Malandraki, O., Mewaldt, R. A., Labrador, A., Stone, E. C., Bale, S. D., Pulupa, M., MacDowall, R. J., Kasper, J. C., Feb. 2020. Properties of Suprathedral-through-energetic He Ions Associated with Stream Interaction Regions Observed over the Parker Solar Probe's First Two Orbits. Astrophys. J. Suppl. 246 (2), 56.

Dierckxsens, M., Tziotziou, K., Dalla, S., Patsou, I., Marsh, M.S., Crosby, N.B., Malandraki, O., Tsiropoula, G., 2015. Relationship between Solar Energetic Particles and Properties of Flares and CMEs: Statistical Analysis of Solar Cycle 23 Events. Sol. Phys. 290 (3), 841–874.

Ding, Z.-Y., Li, G., Hu, J.-X., Fu, S., 2020. Modeling the 2017 September 10 solar energetic particle event using the iPATH model. Res. Astron. Astrophys. 20 (9), 145.

Downs, C., Lionello, R., Mikić, Z., Linker, J.A., Velli, M., Dec. 2016. Closed-field Coronal Heating Driven by Wave Turbulence. Astrophys. J. 832 (2), 180.
Downs, C., Warmuth, A., Long, D.M., Bloomfield, D.S., Kwon, R.-Y., Veronig, A.M., Vourlidas, A., Vršnak, B., 2021. Validation of Global EUV Wave MHD Simulations and Observational Techniques. Astrophys. J. 911 (2), 118.

Eastwood, J.P., Biffis, E., Hapgood, M.A., Green, L., Bisi, M.M., Bentley, R.D., Wicks, R., McKinnell, L.A., Gibbs, M., Burnett, C., 2017. The Economic Impact of Space Weather: Where Do We Stand? Risk Anal. 37 (2), 206–218.

Engell, A.J., Falconer, D.A., Schuh, M., Loomis, J., Bissett, D., 2017. SPRINTS: A Framework for Solar-Driven Event Forecasting and Research. Space Weather 15 (10), 1321–1346.

Erickson, W.C., 1997. The Bruny Island Radio Spectrometer. Pub. Astron. Soc. Aust. 14 (3), 278–282.

Falconer, D., Barghouty, A.F., Khazanov, I., Moore, R., 2011. A tool for empirical forecasting of major flares, coronal mass ejections, and solar particle events from a proxy of active-region free magnetic energy. Space Weather 9 (4), S04003.

Falconer, D.A., Moore, R.L., Barghouty, A.F., Khazanov, I., 2014. MAG4 versus alternative techniques for forecasting active region flare productivity. Space Weather 12 (5), 306–317.

Gallagher, P.T., Lawrence, G.R., Dennis, B.R., 2003. Rapid Acceleration of a Coronal Mass Ejection in the Low Corona and Implications for Propagation. Astrophys. J. Lett. 588 (1), L53–L56.

Gardiner, C.W., 2004. Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences. Springer-Verlag, Berlin.

Georgoulis, M.K., 2008. Magnetic complexity in eruptive solar active regions and associated eruption parameters. Geophys. Res. Lett. 35 (6), L06S02.

Georgoulis, M.K., Bloomfield, D.S., Piana, M., Massone, A.M., Soldati, M., Gallagher, P.T., Pariat, E., Vilmer, N., Buchlin, E., Baudin, F., Csillaghy, A., Sathiapal, H., Jackson, D.R., Alingery, P., Benvenuto, F., Campi, C., Florios, K., Gontikakis, C., Guennou, C., Guerra, J.A., Kontogiannis, I., Latorre, V., Murray, S.A., Park, S.-H., von Stachelski, S., Torbica, A., Vischi, D., Worsfold, M., May 2021. The flare likelihood and region eruption forecasting (FLARECAST) project: flare forecasting in the big data & machine learning era. J. Space Weather Space Clim. 11, 39.

Georgoulis, M.K., Rust, D.M., 2007. Quantitative Forecasting of Major Solar Flares. Astrophys. J. Lett. 661 (1), L109–L112.

Gibson, S.E., Low, B.C., 1998. A Time-Dependent Three-Dimensional Magnetohydrodynamic Model of the Coronal Mass Ejection. Astrophys. J. 493 (1), 460–473.

Gombosi, T.I., Chen, Y., Glocer, A., Huang, Z., Jia, X., Liemohn, M.W., Manchester, W.B., Pulkkinen, T., Sachdeva, N., Al Shidi, Q., Sokolov, I.V., Szente, J., Tenishev, V., Toth, G., van der Holst, B., Welling, D. T., Zhao, L., Zou, S., 2021. What sustained multi-disciplinary research can achieve: The space weather modeling framework. J. Space Weather Space Clim. 11, 42.

Gombosi, T.I., van der Holst, B., Manchester, W.B., Sokolov, I.V., 2018. Extended MHD modeling of the steady solar corona and the solar wind. Living Rev. Sol. Phys. 15 (1), 4.

Gopalswamy, N., Aguilar-Rodriguez, E., Yashiro, S., Nunes, S., Kaiser, M.L., Howard, R.A., 2005. Type II radio bursts and energetic solar eruptions. J. Geophys. Res. 110 (A12), A12S07.

Gopalswamy, N., Yashiro, S., 2011. The Strength and Radial Profile of the Coronal Magnetic Field from the Standoff Distance of a Coronal Mass Ejection-driven Shock. Astrophys. J. Lett. 736 (1), L17.

Guarnieri, V., Tamburini, V., Cougnet, C., Pia, M.G., Spillantini, P., Perino, M.A., Eckersley, S., Heynderickx, D., 2005. REMSIM - Final Report. Alenia Spazio Technical Note. URL: http://www.hep.man.ac.uk/u/johna/pub/Geant4/G4AI/ESA-AO-6765/final%20report%20REMSIM.PDF.

Guo, C., Pleiss, G., Sun, Y., Weinberger, K.Q., 2017. On calibration of modern neural networks. In: Proceedings of the 34th International Conference on Machine Learning - Volume 70. ICML'17. JMLR.org, pp. 1321–1330.

He, J., Rodriguez, J.V., 2018. Onsets of Solar Proton Events in Satellite and Ground Level Observations: A Comparison. Space Weather 16 (3), 245–260.

Heita, P.K.N., 2018. Numerical investigation of solar energetic particle transport between the Sun, Earth, and Mars. Master's thesis, North-West University, South Africa. URL: https://dspace.nwu.ac.za/handle/10394/33865.

Heras, A.M., Sanahuja, B., Lario, D., Smith, Z.K., Detman, T., Dryer, M., 1995. Three Low-Energy Particle Events: Modeling the Influence of the Parent Interplanetary Shock. Astrophys. J. 445, 497.

Heras, A.M., Sanahuja, B., Smith, Z.K., Detman, T., Dryer, M., 1992. The Influence of the Large-Scale Interplanetary Shock Structure on a Low-Energy Particle Event. Astrophys. J. 391, 359.

Hilberg, R.G., 1969. Radiation Protection for Apollo Missions - Case 340. URL: https://www.lpi.usra.edu/lunar/documents/NTRS/collection3/NASA_CR_106949.pdf.

Hoeksema, J.T., Liu, Y., Hayashi, K., Sun, X., Schou, J., Couvidat, S., Norton, A., Bobra, M., Centeno, R., Leka, K.D., Barnes, G., Turmon, M., 2014. The Helioseismic and Magnetic Imager (HMI) Vector Magnetic Field Pipeline: Overview and Performance. Sol. Phys. 289 (9), 3483–3530.

Hu, J., Li, G., Ao, X., Zank, G.P., Verkhoglyadova, O., 2017. Modeling Particle Acceleration and Transport at a 2-D CME-Driven Shock. J. Geophys. Res.: Space Phys. 122 (11), 10938–10963.

Huang, X., Wang, H.-N., Li, L.-P., 2012. Ensemble prediction model of solar proton events associated with solar flares and coronal mass ejections. Res. Astron. Astrophys. 12 (3), 313–321.

Inceoglu, F., Jeppesen, J.H., Kongstad, P., Hernández Marcano, N.J., Jacobsen, R.H., Karoff, C., 2018. Using Machine Learning Methods to Forecast if Solar Flares Will Be Associated with CMEs and SEPs. Astrophys. J. 861 (2), 128.

Iucci, N., Levitin, A.E., Belov, A.V., Eroshenko, E.A., Ptitsyna, N.G., Villoresi, G., Chizhenkov, G.V., Dorman, L.I., Gromova, L.I., Parisi, M., Tyasto, M.I., Yanke, V.G., 2005. Space weather conditions and spacecraft anomalies in different orbits. Space Weather 3 (1), 01001.

Jacobs, C., Poedts, S., 2011. A polytropic model for the solar wind. Adv. Space Res. 48 (12), 1958–1966.

Jeong, E.-J., Lee, J.-Y., Moon, Y.-J., Park, J., 2014. Forecast of Solar Proton Events with NOAA Scales Based on Solar X-ray Flare Data Using Neural Network. J. Korean Astron. Soc. 47 (6), 209–214.

Ji, A., Aydin, B., Georgoulis, M.K., Angryk, R., 2020. All-clear flare prediction using interval-based time series classifiers. In: 2020 IEEE International Conference on Big Data (Big Data). pp. 4218–4225.

Jiggens, P., Heynderickx, D., Sandberg, I., Truscott, P., Raukunen, O., Vainio, R., 2018. Updated Model of the Solar Energetic Proton Environment in Space. J. Space Weather Space Climate 8, A31.

Jiggens, P.T.A., Gabriel, S.B., Heynderickx, D., Crosby, N., Glover, A., Hilgers, A., 2012. ESA SEPEM Project: Peak Flux and Fluence Model. IEEE Trans. Nucl. Sci. 59 (4), 1066–1077.

Jin, M., Manchester, W.B., van der Holst, B., Sokolov, I., Tóth, G., Mullinix, R.E., Taktashvili, A., Chulaki, A., Gombosi, T.I., 2017. Data-constrained Coronal Mass Ejections in a Global Magnetohydrodynamics Model. Astrophys. J. 834 (2), 173.

Jokipii, J.R., 1966. Cosmic-Ray Propagation. I. Charged Particles in a Random Magnetic Field. Astrophys. J. 146, 480.

Jokipii, J.R., Levy, E.H., Hubbard, W.B., 1977. Effects of particle drift on cosmic-ray transport. I. General properties, application to solar modulation. Astrophys. J. 213, 861–868.

Jones, J.B.L., Bentley, R.D., Hunter, R., Iles, R.H.A., Taylor, G.C., Thomas, D.J., 2005. Space weather and commercial airlines. Adv. Space Res. 36 (12), 2258–2267.

Kahler, S.W., 1982. The role of the big flare syndrome in correlations of solar energetic proton fluxes and associated microwave burst parameters. J. Geophys. Res. 87 (A5), 3439–3448.

Kahler, S.W., Cliver, E.W., Ling, A.G., 2007. Validating the proton prediction system (PPS). J. Atmos. Solar-Terr. Phys. 69 (1–2), 43–49.

Kahler, S.W., Ling, A.G., 2017. Characterizing Solar Energetic Particle Event Profiles with Two-Parameter Fits. Sol. Phys. 292 (4), 59.