dataesr/scientific-paragraphs-categorization huggingface dataset:
Acknowledgments: We thank  Jean-Pierre Serre ,  Brian Conrad  and  Matthieu Romagny  for valuable comments on a preliminary version of the paper. We also thank  Brian Conrad  for allowing us to include his alternate proof of Theorem 1. 1,  Skip Garibaldi  for the applications to the abelianization (Cor. 6. 4) and  Laurent Moret-Bailly  for the improvement of Proposition 3. 6.
Recommendations from the U. S. Food and Drug Administration (FDA) on BE study designs for ophthalmic drug products have been previously described. 2, 3 A parallel design is typically used when only one sample is collected per patient. A crossover design may be used when using patients with bilateral disease (i. e. , on both eyes), enabling the collection of two samples (i. e. , one eye per period). As such, PK BE studies of ophthalmic drug products present unique statistical challenges. Certainly, classical NCA is not feasible because it is not possible to calculate individual AUC values from a single concentration. To circumvent this issue, a nonparametric bootstrap NCA has been proposed by Shen et al. 4  This method involves resampling of the subjects, at each sampling time in both reference and test treatment groups, to generate nonparametric bootstrap datasets.
MB-TOST and NCA bootstrap were evaluated on 500 simulated data sets for each scenario. As NCA bootstrap runs faster, it was also evaluated on 10,000 simulated data sets for the challenging scenarios with 10 samples under one null and one alternative hypothesis: P10H 0:1. 25 , P10H 1:0. 9 , C10H 0:1. 25 , and C10H 1:0. 9. NCA bootstrap was evaluated with each of these scenarios, assuming there is no LOQ at 0. 2 mg/L. Whereas MB-TOST was evaluated on scenarios P10H 0:1. 25 and C10H 0:1. 25 with no LOQ at 0. 2 mg/L. In the absence of a defined LOQ, concentrations simulated below zero were set to 0. 0001. MB-TOST and NCA bootstrap were also evaluated on 500 simulated data sets for two parallel design scenarios with 10 possible sampling times and an increased residual variance at b = 30% (P10H 1:0.
Implementation For the NCA bootstrap method, B = 5000 bootstrap samples were performed in the R statistical software version 4. 0. 2. For the model-based approach, the Monolix 2018R2 software  23  was used which implements the SAEM algorithm for parameters. 24 Population parameters were estimated with 300 exploratory and 100 smoothing iterations, ensuring an appropriate convergence of SAEM visually assessed on a couple of data sets. Further, because there was only one observation per subject per occasion, 10 Markov chains were used to improve the precision of the estimation. This choice was motivated by a previous simulation study which already recommended to use 10 Markov chains with three sampling times per subject. 13 Initial values were set at the true value (i. e. , the simulation value [see section simulation settings and Supplementary Material S2]) for all parameters except the treatment, period, and sequence effects all set to zero.
6 Moreover, the model would be subjected to a thorough evaluation prior to its use for the BE analysis. 7 Nonetheless, using the true model and parameter estimates provided a favorable setup for MB-TOST and challenging its robustness to departure from the simulated PK model and parameter values needs to be explored in the future. Another limitation is only 500 datasets were simulated under each of the 16 scenarios due to the limitation of computation power, which can impact the precision of the evaluation of the performance (e. g. , bias, type 1 error rate, and power). Computing times ranged in average from 15 min per data set for parallel designs to 45 min per data set for crossover designs, on a similar computing setup. Notably, it is the default number for simulationbased goodness of fit plots in PsN, a recognized tool for NLMEM.
* M. A. 's research was supported by the  ANR  grant  GATO  ( ANR-16-CE40-0009-01 ) and the  ANR  grant  IsOMa  ( ANR-21-CE48-0007 ). E. F. acknowledges the support of the  ANR  grant  GATO  ( ANR-16-CE40-0009-01 ) and the  ANR grant   Combine  ( ANR-19-CE48-0011 ).
4. In the Appendix we lead a complete discussion of local systems on free loop spaces and their behaviour with respect to the loop product and coproduct. Local systems are unavoidable in the context of non-orientable manifolds  [27, 6] , and also in the context of the correspondence between symplectic homology of T ˚M and loop space homology of M  [26, 6, 5]. On the other hand, they can be useful in applications  [7]. Acknowledgements. The second author is grateful for support over the years from the Institute for Advanced Study, and in particular during the academic year 2019-20. The third author was partially funded by the Agence Nationale de la Recherche, France under the grants MICROLOCAL ANR-15-CE40-0007 and ENUMGEOM ANR-18-CE40-0009.
We have a number of different situations where this model can be used in the real data case. For example in Rex data (field data), where we have a system (a device) with lifetime data and maintenance data matching together. If for example, in this case we have several identical systems and we estimate their lifetime distributions, they are not the same. That means we have some additional random factor. This random factor could be a Markov chain as a random media of our main system. The situation of experimental data is the same. This is the case of the Virkler's data that we present here (see Section 6 ). The same idea is possible considering expert's opinion to describe the lifetime of the device. The random factor here could be the different experience of each expert. A different situation is when we describe a system, i.
Hidden Markov models In a hidden modelling context, there are three basic problems that must be solved for the model to be useful in real-world applications  ( Rabiner, 1989 ). The most difficult one is the training problem, which consists of optimally estimate the parameters of the model from observed data. We usually call the available observations the training dataset. Once the model is constructed (trained), we need to evaluate the model, that is we address the evaluation problem. This involves the calculation of several probabilities associated to the model estimated parameters. Among others we want to score how well the estimated model fit the data, which is useful to choose between competing candidate models. Finally, we are mostly interested in uncovering the hidden part of the model, then we have a decoding problem.
A real case study: The Virkler's dataset As illustrative example we consider the fatigue crack growth problem in a degradation mechanism analyzed in  Chiquet, Limnios, & Eid (2009)  , where a piecewise deterministic Markov process is proposed for the degradation modelling. The data consist of an aluminum alloy specimen that was tested to investigate fatigue crack propagation. 1 tarting from an initial crack of length 9 mm for a particular item in test, the number of cycles for the size of the crack to reach a predetermined value was recorded successively. That is, it is registered the number of cycles every time an increment of size 0. 2 mm in length occurs. The experiment finishes once a critical size of the crack is reached, meaning the failure of the item. The data were first published in  Virkler, Hillberry, & Goel (1979).
The best model will be selected by means of the AIC, defined by AIC = -2 log L + 2(d(d -1) + d(m -1)) , where log L is the estimated maximum log-likelihood function. The results are presented in Table  3  , where it is shown that the 2-states HMM model fits better the data. This conclusion agrees with previous analyses of this dataset where it is argued that fatigue crack growth of this kind of material can be divided into two regimes with different crack propagation rates, see  Abdessalema et al. (2012). Using the fitted 2-dimensional HMM model we can define the reliability of the item in terms of up and down states as well as good and bad signals as explained in Section 3. In our case the hidden state space is E = { 1 , 2 } and we can consider that 1 is denoting a good performance in the system while 2 refers to a bad functioning regime.
Acknowledgements The authors are grateful to two anonymous reviewers and the Editor for many valuable comments and suggestions, which have helped to improve the quality of the article. The authors gratefully acknowledge support from the  Spanish Ministry of Science and Innovation -State Research Agency  through grant number  PID2020-120217RB-I00. This work is supported in part by the  IMAG Maria de Maeztu  grant  CEX2020-001105-M/AEI/10. 13039/50110 0 011033. Funding for open access charge:  Universidad de Granada / CBUA.
This project has received funding from the  European Research Council (ERC)  under the European Union's  Horizon 2020 research and innovation program  under the Grant Agreement No 648132 and from the  ANR de rerum natura   ANR-19-CE40-0018. The fourth author was supported by the  NSF  grants  DMS-1760212 ,  DMS-1760413 ,  DMS-1760448  and the  Lise Meitner  grant  M-2582-N32  of the  Austrian Science Fund FWF. We are grateful for the support from the  NSF  grant  DMS-1952694  that allowed us to work on this project during DART X in New York.
| Stimuli and testing procedure Subjects performed arithmetic problems in two different formats, namely MW and ME conditions (see Figure  S1  for the behavioural results). We selected MW problems with a single operation of addition (Add), subtraction (Sub), multiplication (Mul) and division (Div) from the IL dataset  (Roy & Roth, 2015)  and MW problems with two operations, including addition and subtraction (AddSub), addition and multiplication (AddMul), addition and division (AddDiv), subtraction and multiplication (SubMul) and subtraction and division (SubDiv), from the CC dataset  (Roy et al. , 2015). Each condition consisted of 35 instances. Both of the abovementioned datasets have been widely used in previous machine learning studies on MW problems  (Mandal & Naskar, 2019). For our MW conditions, the original English sentences were translated into Japanese by the first author (T.
at the end of the first day. The presentation order was identical for all subjects. The data from each pair of test runs were averaged to increase the signal-to-noise ratio. Each run contained 45 trials consisting of five trials for each of the nine problems, and the presentation order within each run was randomized. The duration of a single run was 595 s for the MW condition and 455 s for the ME condition. At the beginning of each run, 10 s of dummy scans were acquired, during which the fixation cross was displayed, and these dummy scans were later omitted from the final analysis to reduce noise. We also obtained 10 s of scans at the end of each run, during which the fixation cross was displayed; however, these scans were included in the analyses.
| MRI data acquisition The experiment was conducted using a 3. 0 T scanner (MAGNETOM Prisma; Siemens, Erlangen, Germany) with a 64-channel head coil. We scanned 72 interleaved axial slices that were 2-mm thick without a gap, parallel to the anterior and posterior commissure line, using a T2*-weighted gradient echo multiband echo-planar imaging sequence (repetition time [TR] = 1000 ms; echo time [TE] = 30 ms; flip angle [FA] = 62 ; field of view [FOV] = 192 Â 192 mm 2 ; resolution = 2 Â 2 mm 2 ; multiband factor = 6). We obtained 605 volumes for the MW and RW conditions and 465 volumes for the ME condition, with each set following 10 dummy images. For anatomical reference, high-resolution T1-weighted images of the whole brain were also acquired from all subjects with a magnetization-prepared rapid acquisition gradient echo sequence (TR = 2530 ms; TE = 3.
| fMRI data preprocessing Motion correction in each run was performed using the statistical parametric mapping toolbox (SPM12; Wellcome Trust Centre for Neuroimaging, London, UK; http://www. fil. ion. ucl. ac. uk/spm/). All volumes were aligned to the first EPI image for each subject. Low-frequency drift was removed using a median filter with a 120-s window. Slice timing correction was performed against the first slice of each scan. The response for each voxel was then normalized by subtracting the mean response and scaling it to the unit variance. We used FreeSurfer (https://surfer. nmr. mgh. harvard. edu/) to identify the cortical surfaces from the anatomical data and to register these to the voxels of the functional data. For each subject, the voxels identified in the cerebral cortex (59,499-75,980 voxels per subject) were used in the analysis.
| Motion energy features We employed a Motion energy (MoE) model used in previous studies  (Koide-Majima et al. , 2020; Nakai & Nishimoto, 2020; Nishimoto et al. , 2011)  that can be found in a public repository (https://github. com/ gallantlab/motion_energy_matlab). First, movie frames and pictures were spatially downsampled to 96 Â 96 pixels. The RGB pixel values were then converted into the Commission International de l'Eclairage LAB colour space, and the colour information was discarded. The luminance (L*) pattern was passed through a bank of three-dimensional spatiotemporal Gabor wavelet filters; the outputs of the two filters with orthogonal phases (quadrature pairs) were squared and summed to yield local MoE. Subsequently, MoE was compressed with a log-transformation and temporally downsampled to 0.
| Encoding model fitting In the encoding model, the cortical activity in each voxel was fitted with a finite impulse response model that captured the slow hemodynamic response and its coupling with neural activity  (Kay, David, et al. , 2008; Nishimoto et al. , 2011). The feature matrix F E [T Â 6N] was modelled by concatenating sets of [T Â N] feature matrices with six temporal delays of 2-7 s (T = number of samples; N = number of features). The cortical response R E [T Â V] was then modelled by multiplying the feature matrix F E by the weight matrix W E [6N Â V] (V = number of voxels): b R E ¼ F E W E We used an L2-regularized linear regression with the training dataset to obtain the weight matrix W E. The training dataset consisted of 3630 (3630 s) and 2790 (2790 s) samples for the MW and ME conditions, respectively, whereas, a training dataset was not provided for the RW condition.
The optimal regularization parameter was assessed using 10-fold cross-validation where the 11 different regularization parameters ranged from 1 to 2 10. The test dataset consisted of 605 samples (605 s) for the MW and RW conditions and 465 samples (465 s) for the ME condition (each repeated twice). Two repetitions of the test dataset were averaged to increase the signal-to-noise ratio. Statistical significance (one-sided) was computed by comparing estimated correlations to the null distribution of correlations between two independent Gaussian random vectors with the same length as the test dataset. The statistical threshold was set at P < 0. 05 and corrected for multiple comparisons using the FDR procedure  (Benjamini & Hochberg, 1995). For data visualization on the cortical maps, we used pycortex  (Gao et al.
| Encoding model fitting excluding regressors of noninterest To evaluate the possible effect of sensorimotor features and general cognitive load on the model predictions, we performed an additional encoding model fitting by excluding the features of noninterest. To this end, we concatenated the MoE (visual), BR (motor), RT (cognitive load), accuracy (cognitive load) and letter (orthographic) features. The concatenated features were used as a feature matrix for the encoding modelling. The original training dataset was divided into five subtraining runs and a single subtest run. Encoding models were trained using the subtraining dataset, and the prediction accuracy was calculated using the remaining subtest dataset. L2-regularized linear regressions were applied in the same manner as that described in the 'Encoding model fitting' subsection in the Methods.
| Quantification of format invariance To quantify how the encoding models explained brain activity in each voxel regardless of the presentation format, we defined FI values. In our previous study, we used a similar measurement to quantify how linguistic information is processed in a modality-invariant manner  (Nakai, Yamaguchi, & Nishimoto, 2021). Note that the variance partitioning analysis (de Heer et al. , 2017) cannot be used to analyse the format-invariance in the current experiment because this analysis compares explained variances of two (or more) different types of features extracted from the same dataset. In contrast, this experiment aims at comparing the same operator features extracted from two datasets. To quantify FI based on prediction accuracy, we used the geometric mean of prediction accuracy rather than the weight correlation.
This was justified because models with similar weight values have similar predictive performance. FI consisted of two components: D W and D E. D W was defined as the degree of predictability for the MW test dataset regardless of the training format: D W ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi R EW Á R WW p where R WW and R EW are the intra-format prediction accuracy for the MW model and the cross-format prediction accuracy for the ME model when applied to the test dataset for the MW condition, respectively. Similarly, D E was defined as the degree of predictability calculated for the ME test dataset regardless of the training format: D E ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffi R WE Á R EE p where R EE and R WE are the intra-format prediction accuracy using an encoding model trained with the ME condition and the cross-format prediction accuracy trained with the MW condition when applied to the test dataset for the ME condition, respectively.
| Encoding models predict brain activity in an intra-and a cross-format manner for each subject To confirm that the encoding models successfully captured brain activity during the math problem solving, we performed a series of intra-format encoding modelling predictions and examined the prediction accuracy using a test dataset with the same format as that of the training dataset (Figure  1b , green). First, we trained an encoding model using the MW training dataset and predicted brain activity using the MW test dataset. Using operator features, the encoding model significantly predicted the activity in large brain networks, including the bilateral inferior frontal, parietal and occipital cortices (P < 0. 05, FDR corrected; 40. 1% ± 8. 2% of voxels were significant; Figure  2a  blue; Table  S1 ).
Similarly, we trained encoding models using the ME training dataset and predicted brain activity using the ME test dataset. This encoding model significantly predicted a smaller part of the region, including the bilateral inferior frontal, bilateral parietal and bilateral occipital cortices, compared with that predicted by the MW intra-format encoding model (21. 5% ± 9. 4% of voxels were significant; Figure  2b  blue; Tables  S1 ). To determine whether the encoding models were generalizable to mathematical problems in other formats, we performed cross-format encoding model analyses. Specifically, we applied an encoding model trained with the ME training dataset to the MW test dataset; prediction accuracy was significant in the bilateral inferior frontal, bilateral parietal and bilateral occipital cortices (21.
2% ± 8. 2% of voxels were significant; coloured in orange or white in Figure  2a ; Table  S1 ). Similarly, we applied the encoding model trained with the MW training dataset to the ME test dataset; prediction accuracy was also significant in the bilateral inferior frontal, bilateral parietal and bilateral occipital cortices (12. 2% ± 4. 8% of voxels were significant; coloured in orange or white in Figure  2b ; Table  S1 ). In summary, our encoding models successfully predicted brain activity induced in large brain regions, including the parietal and frontal cortices, across different formats of mathematical problems.
We calculated intraformat predictions using a leave-one-run-out method within the training dataset. Voxels that were consistently predicted in either the MW or ME condition were excluded from FI analysis; we found significant FI values in the bilateral IPS even after excluding these voxels (15. 4% ± 7. 2% of voxels were significant; Figure  S4 ), indicating that the FI values in the abovementioned analyses were not explained by the effect of sensorimotor processing or general cognitive load.
| DISCUSSION To the best of our knowledge, all previous neuroimaging studies on mathematical problem solving have used univariate analysis to assess the format-independency of mathematical problems  (Chang et al. , 2019; Newman et al. , 2011; Sohn et al. , 2004; Zhou et al. , 2018). Thus, it had not been made clear whether the activation overlap of mathematical problems with different formats reflected shared cortical representations. In contrast to univariate analysis, multivariate analysis (i. e. , RSA) can test whether activation patterns are similar across different formats. Notably, training and test datasets had not previously been analysed separately nor had the generalizability of quantitative models to new data been evaluated. In the current study, we demonstrated for the first time that multivariate analyses and encoding model approaches are useful for studying the format-invariance of mathematical problems.
Moreover, some children (and even a few adults) tend to use a counting strategy rather than a memory-based strategy for solving addition problems  (Qin et al. , 2014) , which can be a source of individual variability. Showing a representational similarity between individuals would support the existence of a shared brain mechanisms underlying mathematical problems with seemingly different structures. Although the current study paves the way toward quantitative modelling of complex mathematical problems, we acknowledge that it has several limitations. First, because we chose experimental stimuli from existing datasets  (Roy et al. , 2015; Roy & Roth, 2015) , we could not perfectly balance the number of letters and digit sizes. Although we excluded such effects by subtracting baseline data determined using the RW condition with the same visual stimuli (Figure  3b ) and by explicitly regressing out features of noninterest (Figure  S4 ), the between-condition difference may still have had some influence on the representational organization of the tested math problems.
I G U R E 2 Intra-and cross-format predictions. (a) Comparison of the prediction accuracies when applied to the math word (MW) test dataset using operator features. The intraformat prediction accuracies of the MW model (denoted as R WW , blue) and the cross-format math expression (ME) model (denoted as R EW , orange) are mapped onto the cortical surface of subject ID01. Regions in which activity was predicted, regardless of the stimulus format, are shown in white. Only significant voxels (P < 0. 05, false discovery rate [FDR] corrected) are shown. (b) Comparison between the prediction accuracies of the ME test dataset. Intra-format prediction accuracy, using the ME model (denoted as R EE , blue) and cross-format prediction accuracy, using the MW model (denoted as R WE , orange) are mapped onto the cortical surface.
ACKNOWLEDGEMENTS We thank  MEXT/JSPS KAKENHI  (grant numbers  JP17K13083  and  JP18H05091  in # 4903  [Evolinguistics] for T. N. and  JP15H05311  and  JP18H05522  for S. N. ) as well as  JST CREST  ( JPMJCR18A5 ),  ERATO  ( JPMJER1801 ) (for S. N. ) and  H2020 Marie Skłodowska-Curie Actions  (grant number  101023033  for T. N. ) for their partial financial support of this study. The funders played no role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript.