dataesr/scientific-paragraphs-categorization huggingface dataset:
The coefficients of the linear functions of all the neurons are tuned by an optimization algorithm. This phase is called the training. We use a variant of the stochastic gradient descent method (the Adam optimizer) to minimize the mean square error between data and predictions. Neural networks are accurate for regression problems and extend well to a huge dataset or to high dimension problems. One difficulty is that such a model involves a lot of hyperparameters, and many combinations of these hyperparameters may give low accuracy results. For example, we have to choose the architecture (number of layers and neurons per layer), the initialization of the linear coefficients, the optimization algorithm, the number of epochs (iterations of the algorithm) and batches (splitting of the data to calculate gradients in the stochastic gradient descent).
Cross-Validation Each method depends on some key smoothness parameters (usually called hyperparameters) that need to be tuned properly to get a good performance. This is done via crossvalidation. K-fold cross-validation consists of breaking down the training data into K folds {F k : k = 1, 2,. , K}, and for a given candidate parameter, the corresponding model is constructed using as training set the K -1 folds where the remaining fold is treated as a validation dataset. Thus, for a given value of parameter β, the corresponding model f is trained K times (K different combination of K -1 folds choosing from the total K folds). We then measure the performance of f at the choice of parameter β using the cross-validation error defined by CVE β 1 K K k 1 xi∈F k f x i ( ) -y i 2 In the particular case where each data subset only contains one single observation, the method is called leave-one-out cross validation.
Roughly speaking, this provides the average performance of f associated with the parameter β on K different unseen folds of the training data. The parameter β minimizing this cross-validation error would be a suitable one to be used as a global parameter in predicting the real testing dataset. For k-nearest neighbors, kernel regression, regression tree, bagging, and random forest, a 10-fold cross validation was used. For thin plate splines, the penalty coefficient is estimated through generalized cross validation, which may be regarded as an approximation to leave-one-out. For the neural networks, the training data set was randomly cut in 3 parts: 80% for the training, 10% for checking over-fitting during the training, and 10% for selecting the final network. After that hyperparameters selection, all results presented in this article are obtained on a huge separated test dataset, as showed in Table  2.
Results for the Storm of March 2013 The numerical results reported in the following tables and figures present an analysis of the distribution of the errors: e i y i -ŷi , for i = 1, 2,. , n, for the different investigated methods, where y i = log (D αα ), ŷi is the prediction of the considered model, and n is the size of the considered test dataset. We train our models on the TRAIN_M13 data set, containing 20,160 samples. The TEST_M13 test data set is detailed in pitch-angles and energies but contains only 4 discrete L-shells values (L = 2, 3, 4, 5). The TEST_M13_L data set is however detailed in L and contains L-shell values regularly spaced from L = 1. 6 to L = 5. 2 by 0. 1 step (37 values) and a few angles and energies values. These datasets are sampled on a grid and there is no uncertainty on the points.
The DNN method in Figure  3  (right) seems to us the most capable for this difficult exercise, which confirms the global metrics of Table  5. We conclude that without any prior assumption on a physical phenomenon and on the database, it is difficult to advise the use of a particular machine learning model. One main reason is the data used to train the model have a big influence on the model performance, which makes it hard to generalize the capabilities of a given model. Here, we believe the different series of tests and comparisons are explicit enough to conclude that the DNN method is a good candidate to perform the rest of the study and to generate a more global model.
Results for the Global Model of Pitch Angle Diffusion During HSS Storms In this section, we use the data from 32 storms in order to build a database of statistical event-driven diffusion coefficients that is embedded within a machine learning model for facilitating its use. The method relies on constructing first an averaged model and then using the deep neural network (DNN) previously selected to learn and output the solution of the averaged model. As in the previous sections, we will see the machine learning model does not entail any issue to interpolate and reproduce the averaged model. Questions arise more about the physical choices we make to build the averaged model (cf. discussion below and in Section 4. 1). Interestingly, the machine learning model was of great help for the various investigations we conducted.
As the training step was quite fast (based on the knowledge acquired during the March 2013 storm study), we could test different ways of manipulating and averaging the data when iterating to choose how to best parametrize the statistical model. Another strength of the machine learning approach is the simplicity of performing comparisons with the model since it delivers continuous maps of the solution with a simple numerical subroutine able to output a 5 to 6 dimensions solution. On the contrary, manipulating directly the database and using discrete points is very constraining. It can also be source of direct errors or interpretation errors when it is a given plotting software (e. g. , Python subroutines) that carries intrinsic ad-hoc interpolation with integrated smoothing procedures.
Training the DNN Global Model The data used to generate the global model is DS3_AVG described in Section 2. 1. 4 with 2. 3e6 data points. We then use TRAIN_AVG (2. 3e5 data points), unless specified differently for training and validation, and TEST_AVG (2. 3e5 data points) for test. The training of the global model D αα (K p , t, L, E, α) was not harder than the model previously trained for the storm of March 2013 in Section 3. 1. The two more dimensions of the input space entailed a larger neural network. The bigger amount of data (from 20,000 to 230,000) caused a longer training. Generating the whole model took a few hours of computation on a standard computer. For comparison the simple generation of a mean model, without machine learning and performing only means throughout the whole database, took a few days on the same computer.
Again for comparison, computing 19. 3M diffusion coefficients for around 10-days event takes around 15,600 h spread over 1,300 processors on a CEA massively parallel supercomputer  [17]. This brings another advantage of machine learning methods to be able to manipulate simply and at low cost a large database, with the possibility to operate on them basic statistical operations useful for the understanding of the database. In Figure  4  (left), we represent the mean average error (MAE), the mean square error (MSE), and one minus the explained variance score (EVS) computed when the model is evaluated against the TEST_AVG test dataset (230,000 data not used during the training phase performed with the TRAIN_AVG dataset). Because the dataset contains little noise, we can train neural networks going deep, with depths of the network going from 6 to 11 hidden layers on the x-axis.
We see an optimum of low values of the three metrics is found for a depth of 9. The same three quantities are plotted in Figure  4  (center) using different sizes of training dataset (from 1 to 15% of the TRAIN_AVG dataset), with DNN of depth 8 or 9. From these results, we selected the neural network of depth 9 trained with 10% of the data. As over-fitting is not an issue here, we could reach better accuracy by taking a more important capacity for the model, or just by taking more epochs as discussed next. This is not obvious on Figure  4  as with a higher depth, error is growing (after depth 9), but it is possible to be more accurate by varying all hyper-parameters. However, we have also seen that the loss of FIGURE 4 | Errors calculated on the dataset TEST_AVG are plotted for different DNN models, with various depth on the left, and various sizes of training dataset on the middle.
On the left, models are trained on 10% of the data, meaning around 230,000 data. On the middle, dot lines with circles are for a model of depth 8, and continuous lines with crosses for a model of depth 9. In blue, we plot the Mean Average Error, in red the Mean Square Error, and in black one minus the Explained Variance Score. On the right, the loss function (MSE) is plotted during the training, evaluated on the training dataset, and on the validation dataset. We see at the end that we make more often evaluations of these errors, and the training stops selecting the more accurate model in the last epochs. Note that this loss may not be compared with anything in this article, as it is given on scaled data. accuracy due to the DNN model is less the issue than the loss of accuracy caused by an averaged statistical approach (cf.
discussion in Section 3. 2. 2). Figure  4  (right) represents the loss function during the training and the validation phases of the model of depth 9 over 10% of the data. The loss function is the minimal MSE computed over all the data and evolving according to the epoch number, which represents the number of cycles the data are used in a training or validation step. After 900 epochs, we evaluate more often the loss function, because we stop at the best loss value obtained on the validation dataset.
The K ponly model is too rough and causes too much error as we will discuss next, and thus D αα (t, K p , L, E, α) was retained. Here, time could be interpreted as representing any other geomagnetic index (or some global measure of them). Similarly, one could have use 2 (e. g. , Dst and K p ) or 3 (or more) geomagnetic indices and their history (Dst* = max 24 h (Dst(t)), Kp* = max 24 h (K p (t))) or characteristic quantities (such as solar wind velocity, dynamic pressure, etc. ) so that the problem becomes single valued, without averaging. In principle, one could also use all wave parameters as entry parameters of the unitary diffusion coefficients D αα (t, L, E, α) since they were used for the generation of the single diffusion coefficients. In that case, the complexity of merging and coupling correctly various complex database together becomes an issue.
On a K p -Only Model Before the retained average model presented above, we tried a simpler model, based only on K p , i. e. , D αα (K p , L, E, α), as the modeling of pitch-angle diffusion is not time-dependent in most common space weather simulations and follows only the dynamics of a single index, such as the K p or AE index. Interestingly, D αα (K p , L, E, α) can be obtained in three different ways: averaging the whole data DS2 over times and storms, averaging DS3_AVG over time (cf. Section 2. 1. 4), or by averaging the machine learning model over time. The two first methods require to run through the dataset many times and to select the right data in order to perform the proper averages. These operations are prone to errors. On the other hand, averaging the DNN model is extremely seductive because it is immediate and simple to perform.
Then, it is straightforward and more accurate to build a DNN model for each of the two components: one for the n-cyclotron and one for the Landau component. The full model is then made by the sum of both models. The only drawback is the increase of the memory storage by a factor 2. Finally, machine learning models provide a wide and continuous model in a high dimensional space, which can produce extrapolation and surprising results (right or wrong) in particular for rare events and in the various high-dimensional corners of the model. These solutions always require for verification to go back to the database and to explore it more and more to the point of knowing (or trying to know) the data in all its aspects. This is often very time consuming, if not practically impossible, even if facilitated by the machine learning method in use.
These difficulties call for reliable and robust testing methods and metrics to be able to rely more and more on the machine learning method with less and less verification of the database. In this work, even though the DNN model has shown a good accuracy, we do not think we have yet reached this level as, for instance, there are some remaining issues due to strong gradients (e. g. , associated to Landau diffusion) or there is no possibility to verify and validate the behavior of the model for special configuration (e. g. , low K p in the first time of the HSS storm). The second point may call for using a given mean model simultaneously with its variance, which signifies using DNN that propagate the distribution of the data. A mean answer would be given with a confidence index based on the variance.
CONCLUSION In this work, we consider 8 nonparametric methods of machine learning based on local evaluation (k-nearest neighbors and kernel regression), tree-based methods (regression tree, bagging and random forest), neural networks, and function approximations (Radial basis and splines). With them, we derive machine learning based models of event-driven diffusion coefficients first for the storm of March 2013 associated to high-speed streams. We present an approach that exhibits some selected properties of the machine learning models in order to select the best method for our problem among the 8 methods. The approach is based on 3 diagnostics: compute the main global metrics (including mean, median, minimum, maximum, standard deviation, and quartiles errors) at various resolution of the database, generate violin plots for analyzing the error distribution, and compute the correlation of each method with the other to enlighten their differences and exhibit the main families.
Machine learning methods and the easily accessible numerical procedures that favor their use have a wide potential for the type of problems we presented, whether it is for manipulating, interpolating, representing, or for analyzing a huge database of event-driven diffusion coefficients and, more generally, database of diffusion coefficients combined with the main parameters used to compute them, such as plasma density and wave parameters. An inherent drawback is the human involvement required to analyze these huge databases in order to potentially identify regions of model deviance or model breakthrough. The DNN method that is proposed here has the advantage to be extended to more parameters characterizing storms (including OMNI solar wind and geomagnetic index data), which should improve the accuracy and predictability of global models.
DATA AVAILABILITY STATEMENT Publicly available datasets were used in this study. EMFISIS data was obtained from https://emfisis. physics. uiowa. edu/Flight/. Data are also available at NASA CDAWeb https://cdaweb. gsfc. nasa. gov/index. html/. Datafiles and both RBF and KerReg models are hosted on https://github. com/ ML-Space-Weather/ PADiffusion-HSS. For DNN models, we used DJINN package https: //github. com/LLNL/DJINN. It is based on scikit-learn and tensorflow library. The tree-based and local evaluation methods are based on sklearn library. We use both the tree and the neighbors functions for Regression tree and KNN, respectively. We use both BaggingRegressor and RandomForestRegressor functions of ensemble module (sklearn. ensemble) of sklearn, respectively. And we use KFold function of model selection module from sklearn to perform K-fold cross validation in selecting the hyperparameters of the methods.
Acknowledgments M. S. and I. M. wrote the paper. M. S. reviewed the literature on complexity and policy processes. M. S. and I. M. developed the baseline policy process model. I. M. implemented the model and analyzed the results. I. K. implemented an earlier version of a baseline model of policy processes. K. S. , I. K. , J. F. , and  G. D. M. S. provided conceptual and technical comments on the paper. e authors thank  Jacob Arbeid ,  Karsten Donnay , and  Chiara Gerosa  for additional comments and edits, as well as  Didier Wernli  and the reviewers for their useful suggestions. I. K. acknowledges support from the  ERC   Horizon 2020 Research and Innovation Programme  under grant  725594  (time-data).
TESS OBSERVATIONS, CANDIDATE IDENTIFICATION, AND VETTING LHS 1678 (TIC 77156829, TOI-696, L 375-2, LTT 2022, NLTT 13515) was observed by TESS in Sector 4 from UT October 19 to  November 14, 2018 (25. 95 d)  in CCD 1 of Camera 3 and in Sector 5 from UT November 15 to  December 11, 2018 (26. 31 d)  in CCD 2 of Camera 3. LHS 1678 data were collected at 2-min cadence because the star was prioritized for high-cadence measurements as part of the Cycle 1 Guest Investigator Program G011180 1  , the Cool Dwarf target catalog  (Muirhead et al. 2018)   2  , and the TESS Candidate Target List (CTL,  Stassun et al. 2018b). LHS 1678 was also observed in Sectors 31 and 32 at 2-min cadence, after the majority of the analyses here were performed. We briefly discuss preliminary findings from the full 4-sector TESS dataset in § 7.
1. Aside from the aforementioned section, we focus our analyses on TESS Sector 4 and 5 data only. TESS 2-min cadence data were processed into target pixel files (TPFs) and subsequent light curves by the NASA Ames Science Processing Operations Center pipeline  (SPOC, Jenkins et al. 2016). The light curves revealed a star with little variability above the TESS noise level (indicating a lack of surface activity such as star spots) over nearly two months of continuous observations and exhibited no evidence for flares. The light curves were searched for periodic, transit-like signals using the Transiting Planet Search module (TPS,  Jenkins et al. 2010 ) and revealed two candidate planet signals that passed a series of data validation tests  (Twicken et al. 2018; Li et al. 2019). The TESS pipeline identified two candidates, TOI-696.
Following this lead, we examined the SPOC Data Validation Reports for TOI-696. 02 and found similar evidence for excess signal at 1/4 the originally-identified period in the phased light curve. We then performed two independent planet candidate searches in an attempt to recover the TPS candidates at the same periods. Using both a modified version of the Quasi-periodic Automated Transit Search  (QATS, Carter & Agol 2013; Kruse et al. 2019)  and Transit Least Squares (TLS,  Hippke & Heller 2019a,b)  search pipelines, we identified transiting planet candidates with periods of 0. 86 days and 3. 69 days, recovering TOI-696. 01 at the same period and identifying TOI-696. 02 at 1/4 the original period, consistent with the excess signal identified in our vetting and validation efforts. With the SPOC team, we investigated the details of the TPS candidate search to reconcile the period difference for TOI-696.
Low Magnetic Activity and Rotation: The TESS light curve spanning two sectors showed no signs of magnetic activity, either in the form of spot modulation or flares. We also extracted archival data from the All-Sky Automated Survey for Supernovae (ASAS-SN,  Shappee et al. 2014; Kochanek et al. 2017; Jayasinghe et al. 2019 ) using the online tool 5  and the Gaia DR2 J2015. 5 coordinates. The Lomb-Scargle (LS) periodograms of the 3. 9-year V and 6. 7-year g band datasets show no signs of rotation modulation (Fig. 2 ), in line with our TESS findings, and only single-point outliers that cannot be confirmed as flares without higher cadence data. Our 16-year CTIO/SMARTS 0. 9m dataset discussed in detail in § 3. This is slower than the vast majority of M dwarf rotation periods to date; we note that this relation is not calibrated to such a low value of log R HK and this estimate is highly imprecise.
Astrometric Detection of a Stellar or Substellar Companion Sixteen years (2004-2020) of ground-based astrometry from the RECONS program at the CTIO/SMARTS 0. 9m telescope in Chile show compelling evidence that LHS 1678 hosts a low-mass stellar or substellar companion with an orbital period longer than the timespan defined by the dataset. As initially reported by  Jao et al. (2017) , there is residual motion in the position of the LHS 1678 photocenter after solving for parallax and proper motion. Images of the field were acquired through the V J (henceforth simply V ) filter at air-masses < 1. 1, meaning that corrections required for differential color refraction were minimal. Following the prescription described in  Jao et al. (2005) , the position of LHS 1678 was measured relative to five reference stars in 136 images taken on 30 nights with seeing better than 1.
8. Once a preliminary parallax and proper motion were measured, we subtracted the corresponding model from the data to get residuals in R. A. and Decl. and fit an orbit to these residuals. Our methodology here is a least-squares approach to constraining orbital elements, as described in, e. g. ,  Winters et al. (2017) , guided by the work of  Hartkopf et al. (1989). We adopt a set of starting values for the orbital period, epoch of periastron, and orbital eccentricity. We iteratively calculate an orbit and compare with the data, tweaking the starting values until we identify a best-fitting orbit. To derive our final parallax and proper motion, we subtracted the orbit model from the data and measured parallax and proper motion using the new dataset. We derive a trigonometric parallax of 52.
However, bound sources beyond the resolution limits of the archival imaging (∼1 ) that are co-moving with LHS 1678 could still remain. Such companions could be eclipsing binaries (EBs), host the transiting bodies, or add significant flux to the light curve and dilute the transit signals, leading to a bias in the derived planet radii  (Ciardi et al. 2015; Furlan & Howell 2017). To identify close-in bound companions, we observed LHS 1678 via both speckle imaging using the SOAR HRCam+SAM setup and adaptive optics using VLT NaCo. LHS 1678 was observed three times with the High Resolution Camera (HRCam) on the SOAR Adaptive Optics Module (SAM,  Tokovinin et al. 2016 ) at the Southern Astrophysical Research (SOAR) 4. 3m telescope. Optical speckle data were collected on UT 2019 November 10, UT 2019 December 13, and UT 2020 January 7 in the I-band (λ c = 824 nm / 170 nm), which is approximately centered on the TESS bandpass (Table  3 ).
Ground-Based Time-Series Photometry To re-detect the transit signals attributed to the TESS planet candidates, refine transit depths and ephemerides, and rule out nearby sources as contaminating nearby EBs, we pursued seeing-limited ground-based time-series observations. These observations were organized, obtained, and analyzed through TESS Follow Up Observing Program (TFOP) 8  Subgroup 1 (SG1). The observations were scheduled using the TESS Transit Finder, a modified version of the Tapir software  (Jensen 2013). A summary of the observations obtained through SG1 for each candidate planet transiting LHS 1678 is provided in Table  4. This includes observation dates and parameters and key analysis results for each ground-based time series. LCOGT -We obtained ten transit observations of the inner planet candidate (TOI-696.
c "Late" denotes that the event was detected later than the time predicted by the ephimerides. tected at the anticipated times and with the expected depth (although with low signal-to-noise, SNR), one partial transit of the outer planet candidate was detected, and the possibility of a nearby EB causing the apparent TESS transits was ruled out at the periods of both planet candidates. PEST -A single transit of the inner planet candidate was observed using the 0. 3m Perth Exoplanet Survey Telescope (PEST) 9. PEST carries an SBIG ST-8XME CCD camera with a plate scale of 1. 2 pixel -1 and a 31 × 21 field of view. The observations were performed in the R c filter. Image reduction, differential photometry, and light curve analyses were performed using the custom C and python based PEST pipeline 10.