DOI: 10.5281/zenodo.10276910
title: List of near-Mercury regions predicted by an LSTM machine learning model during the MESSENGER era

description: Output of the Smith & Jackman et al. [submitted, JGR, 2023] supervised machine learning approach to classify near-Mercury regions. The list spans the full near-Mercury exploration by the MESSENGER spacecraft from March 23rd 2011 to April 30th 2015, with 1 entry per second. The machine learning model is a Long Short Term Memory (LSTM) Recurrent Neural Network (RNN), trained on the list of magnetospheric regions which is available here: Sun, 2023 https://zenodo.org/records/8298647 and is discussed in Sun et al., 2020. We refer to this list as S2020 hereafter and it includes manual labels for the three key regions of (1) magnetosphere, (2) magnetosheath, (3) solar wind and also for two key transition regions of (4) magnetopause) and (5) bow shock.
Our LSTM model returns probabilities for three key regions of (1) magnetosphere, (2) magnetosheath, (3) solar wind. We ran the model on a test set of several months of complete MESSENGER orbits and results are shown in the paper. For the full 4-year duration of MESSENGER's near-Mercury exploration, we had very high confidence in our model performance far from the "transition regions" near magnetospheric boundaries of bow shock and magnetopause. Thus we only ran the full model on transition regions and we distinguish between the predictions/flags in these cases by defaulting to the S2020 column label when the LSTM hasn't made a prediction. LSTM-predicted regions are labelled (1)=Magnetosphere, (2)=Magnetosheath, (3) Solar Wind, and the cases where the predictions default to the S2020 manual labels are marked (1a)=Magnetosphere, (2a)=Magnetosheath, (3a) Solar Wind.
When the LSTM is run on a given time step, it returns a probability, each of which are listed in the output file. The region which the highest probability is assigned as the model prediction. We further add a model confidence flag of (1) for good and (2) for poor. These flags are based on the criteria outlined in the Smith&Jackman et al. [submitted, 2023] paper. The key criteria are that (a) a region probability must exceed 0.9, (b) other two region probabilities must be significantly distinct from the highest probability, with values below 0.8, (c) anything inside of 1.33 RM is magnetopause and outside of 7.15 RM is solar wind (based on extensive empirical examination).
 
The dataset presented here includes the following columns:

Date [Day:Month:Year Hour:Minute:Second]
S2020 label [1 = Magnetosphere, 2=Magnetosheath, 3=Solar Wind, 4=Magnetopause, 5=Bow Shock]
LSTM Model Probability (1) Magnetosphere
LSTM Model Probability (2) Magnetosheath
LSTM Model Probability (3) Solar Wind
LSTM Model Predicted Region (1)=Magnetosphere, (2)=Magnetosheath, (3) Solar Wind
Model quality flag (1) = Good, (0) = Poor
 
Here is an example of an entry in the list:
Date S2020 prob_MSP prob_SH prob_SW lstm_prediction qf
26/03/2011 01:14 2 3.81E-07 8.90E-09 1 2 1
 
We suggest that users can calculate their own Ground Truth Flag to compare against any given manually labelled list of near-magnetosphere regions at Mercury. We note that S2020 (and similar lists) generally label 5 regions whereas our model returns just 3 possible region predictions.
 
The current version is updated on December 6th 2023
This work was supported by Science Foundation Ireland Grant 18/FRL/6199  (PI Caitriona Jackman), Irish Research Council Laureate Consolidator Award (SOLMEX, PI Caitriona Jackman), NASA Discovery Data Analysis Program (DDAP) Grant #80NSSC22K1061 (PI Weijie Sun), NSF Grant 2321595 (WeiJie Sun), and NERC Independent Research Fellowship NE/W009129/1 (PI Andy Smith).