dataesr/scientific-paragraphs-categorization huggingface dataset:
If the resulting output is higher than the set threshold, the neuron is considered as being activated, otherwise not. Lastly, the hidden layer leads to the final output result, displayed in the output layer. Optimization of ANNs is ensured through the back-propagation method  25  in the NeuralNet package and the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method  26  in the Nnet package. For detailed information on ANN functioning, see  27. In the ANN models, the inputs are the activities of each enzyme (PGAM, ENO and PPDK) used in the in vitro experiment (Table  S1 ,  8  ), and the output is the predicted pathway flux (J pred ). Also, each weight in the ANN is assigned automatically by RStudio. Given the small amount of experimental data (Table  S1 ), ANN models are built with a training set made up of the complete Tables S1 or S2 datasets (the data from the experimental dots or data from the fitting curves, respectively), then optimized through a Leave-One-Out cross validation (LOOcv) procedure.
Enzyme Kinetic equations a PGAM v = V f [3PG] K m3PG -Vr [2PG] K m2PG 1+ [3PG] K m3PG + [2PG] K m2PG + [PP i ] K iPP i ENO v = V f [2PG] K m2PG -Vr  [PEP]  K mPEP 1+ [2PG] K m2PG + [PEP] K mPEP + [PP i ] K iPP i + [3PG] K i3PG PPDK b v = V f ABC-PQR Keq KmAB+KmBA+KmC B+KmBC+ V f K mQ P Vr Keq + V f K mP Q Vr Keq + V f K mQ R Vr Keq + V f K mR Q Vr Keq +ABC+ V f PQR Vr Keq K i_ATP 140 K ii_PPi a 1,000 K i_PEP a 1,000 K i_AMP a 1,000 K PPi_AMP a 1,000 K i_PPi a 1,000 with E the calculated error, P the tested parameter, ω j is the calculated weight for each experimental data column, x i,j a point in the dataset and y i,j (P) the corresponding predicted value; i and j are the rows and columns in the experimental dataset. The weight calculation method was the mean square: ω j = 1 x 2 j , with x 2 j the mean of squared data from one column.
The software provides a list of optimization methods, to find optimized values for the estimated parameters (https ://copas i. org/Suppo rt/User_Manua l/Metho ds/Optim izati on_Metho ds/). Again, two types of estimations are carried out: • one estimating one or several parameters with one target value and • the other estimating one or several parameters with many target values. The models obtained constitute the white-box approach, with known enzymatic parameters and equations. Grey-box approach. In the specific case of the grey-box approach, to improve the COPASI model predictions, the kinetic equation of PPDK is changed to a ter-reactant reversible equations  8  which was modified as follows (  6 ): with the adjustment term α V f -V f 0 in the denominator, α is a defined number, V f 0 is the PPDK maximum rate in the forward direction used in the in vitro reconstitution and V f is the PPDK maximum rate in the forward direction in the model.
Model comparison. To compare accuracy of the models, RMSE, R 2 and AIC are assessed for the experimental dataset (Table  S2 ). The same statistical metrics are used to evaluate their generalization ability with the test dataset (Table  S3 ). AIC measures the quality of the model by taking into account its complexity. Additionally, as the ratio "number of data-number of parameters" is less than 40, a corrected AIC is calculated as follows  20, 34  : with k being the number of parameters, SSE the Sum of Square Errors and n the number of data. Furthermore, to assess the generalization ability of the models, a comparison of RMSE, R 2 and MAE is made on the previous test set (Table  S3 ).
Application and results ANN modeling of the second part of glycolysis. First, we model the second part of E. histolytica glycolysis using the black-box modeling approach with ANN models and the first experimental dataset (Table  S1 , Fig. 4A, B ) or the second experimental dataset (Table  S2 , Fig. 4C-F ). For the first dataset, the evaluation of RMSE in cross-validation (cvRMSE) and MAE in cross-validation (cvMAE) shows a fluctuation of the error values when the number of HUs is varied and allows the identification of the best ANN model, presenting the lowest cvRMSE and cvMAE values. Also, the calculation of N h gives a maximum value of 4 ( α = 2), making it possible to identify the best model, regarding cvRMSE and cvMAE, with a number of HU equal to 1 (Fig. 4A ). By comparing the ANN predicted fluxes with the experimental ones, we observe that this model can predict rather well the flux of the pathway for the training set, especially at high values of flux (Fig.
4B ), and even if the calculated errors remain high (cvRMSE = 4. 23 nmol•min -1 , cvMAE = 2. 78 nmol•min -1 ). The prediction of the test set shows that the model predicts the flux better when PGAM or ENO activity is varied, than when PPDK's activity is varied. This can be explained by the small experimental data number in the training set, which is derived from experimentally controlled conditions. We built other ANN models with the NeuralNet package and tanh activation function and Nnet package, but the predictions are less good than those of previous models, with lower R 2 in cross-validation (cvR 2 ) and respectively, cvRMSE = 4. 47 nmol•min -1 and cvMAE = 2. 84 nmol•min  -1  , for the first one and cvRMSE = 4. 56 nmol•min -1 and cvMAE = 2. 66 nmol•min -1 for the second one (Fig. S1 ).
Afterwards, we built another ANN model, this time using the second dataset, corresponding to the data from the fitting curves obtained from the experimental points in the first dataset. From the two packages used, we notice that, with NeuralNet and tanh activation function, it is easier to identify the optimal number of HUs, which is 18, but this is not the case with the Nnet package, where the models with 22 and 23 HUs present a better cvMAE or a better cvRMSE (Fig. 4C, D ). As RMSE is the most used model selection criterion of both, we use 23 HUs for the second model with the Nnet package. The comparison of these two models shows their ability to simulate the metabolic pathway, with better results for the Nnet model (Fig. 4E, F ). Also, the calculation of N h gives a maximum value of 23 ( α = 2); thus, both models comply with the limit set by the equation.
However, in order to select the best model and ensure that it is not too specific to our second dataset, we used the test set from the most performing COPASI model (Table  S3 ), and predicted the final flux with our two ANN models. The NeuralNet model produced better results, with RMSE = 1. 61 nmol•min -1 and MAE = 1. 37 nmol•min -1 , compared to the Nnet model. These results suggest that this novel black-box approach, using ANN, is relevant for constructing metabolic pathways from experimental data, with better predictions when working with bigger datasets, whether it be with NeuralNet or Nnet package. Design of metabolic network with the white-box approach. After the modeling phase using the black-box method approach, we focused on the white-box approach and designed mechanistic models with COPASI.
Factors impacting model performance. During this study, we relied on three main statistical metrics (RMSE, MAE and AIC) to evaluate model performance. The results revealed that different criteria are important and impact the value of these metrics and thus the model performance itself. Among these criteria, we identified the size of the dataset, but also the choice of the activation function (log or tanh) and the number of HU in our ANN models. Indeed, having a large number of high-quality datasets is essential to obtain a good ANN, and one challenge here would be to avoid over-fitting  54, 55. Other studies bring out the importance of the size of the input sequence during the analysis of the DNA sequence, to increase model performance  56. Also, they reveal the relevance of neural network architecture, proposing the design of multi-task neural networks with multiple output variables  57.
Data availability The datasets used in this study are fully included and described in the Additional file. The ANN and COPASI models built during the present study are available in the Github repository, https ://githu b. com/ophel ielt/Lo-Thong _et_al. _White -box_grey-box_and_black -box_pathw ay_model ing. git. All data generated or analyzed during this study are included in this published article (and its Supplementary Information files).
The capture of a target spacecraft by a chaser is an on-orbit docking operation that requires an accurate, reliable, and robust object recognition algorithm. Vision-based guided spacecraft relative motion during close-proximity maneuvers has been consecutively applied using dynamic modeling as a spacecraft on-orbit service system. This research constructs a vision-based pose estimation model that performs image processing via a deep convolutional neural network. The pose estimation model was constructed by repurposing a modified pretrained GoogLeNet model with the available Unreal Engine 4 rendered dataset of the Soyuz spacecraft. In the implementation, the convolutional neural network learns from the data samples to create correlations between the images and the spacecraft's six degrees-of-freedom parameters.
The experiment has compared an exponential-based loss function and a weighted Euclidean-based loss function. Using the weighted Euclidean-based loss function, the implemented pose estimation model achieved moderately high performance with a position accuracy of 92. 53 percent and an error of 1. 2 m. The in-attitude prediction accuracy can reach 87. 93 percent, and the errors in the three Euler angles do not exceed 7. 6 degrees. This research can contribute to spacecraft detection and tracking problems. Although the finished vision-based model is specific to the environment of synthetic dataset, the model could be trained further to address actual docking operations in the future.
The relative position and attitude data were considered real-time navigation data and accomplished with the Clohessy-Whiltshire relative motion equations to estimate the rendezvous trajectory of the ATV to the international space station. In addition to deep learning algorithms, several methods did not attempt to process the entire image; instead, they utilized the image only for feature detection. Liu et al. [9]  applied the edge detection algorithm to extract meaningful features from the images of a cylinder-shaped spacecraft. The ellipses, which are obtained from arc detection, were employed to estimate spacecraft poses by the manipulation of shape, size, and position of features. For a similar method, Aumann  [10]  developed a pose estimation algorithm using Open source Computer Vision (OpenCV) to detect two longitudinal lines on the sides of a cylinder-shaped object.
The regression method demonstrated in  [14]  derived equations for constructing convolutional neural network models. This study used various orientation estimation models for rotation in different dimensions. According to the methodology, the estimation algorithms for viewpoint estimation, surface-normal estimation, and 3D rotation have different rotation parameters and operations. Spacecraft usually behave as 3D rotating objects. Thus, the implementation of spacecraft pose is beyond the determination via Euler angles, as shown in surface-normal estimation. Instead, quaternions are required to represent the object's angle of rotation. Many public datasets contain images with labeled data that are positioned and oriented in a representation of quaternions. Proença and Gao  [15]  generated a dataset by using Unreal Rendered Spacecraft On-Orbit (URSO).
The tool proposed in that study is a simulator built in Unreal Engine 4 that creates realistic images of the spacecraft surroundings by mimicking the appearance of outer space. The generated images can visualize these outer space conditions for the spacecraft under harsh lighting conditions and can use realistic earth-surface images as the background. They also demonstrated a method that uses a ResNet architecture based on a pretrained CNN model as a backbone. This method achieved high accuracy but also has high complexity. Consequently, it consumes large amounts of computational resources. Another previous work on spacecraft datasets can be found in  [16] , which introduced the spacecraft pose network (SPN), a custom CNN whose architecture includes three separate branches. They trained the CNN model using a public dataset, Spacecraft PosE Estimation Dataset (SPEED) and estimated the six degrees of freedom parameters separately.
The position was estimated from a 2D bounding box on a target detected by one branch of the CNN model using the Gauss-Newton algorithm. The relative attitude was determined directly from the other two branches using a hybrid discrete-continuous method. Although the custom convolutional network is beyond the research scope, it provides a significant contribution and performs estimation using separated parameters classification and regression method. Kendall et al. [17]  presented a deep neural model that employed a convolutional neural network to perform pose estimation for a camera. The dataset preparation process considers the pose as parameters relative to the scene and a practical algorithm for pose estimation is developed. The researchers implemented this process using a modified GoogLeNet architecture, which is a CNN model developed by Google.
The pose estimation model was initially trained with interior data and subsequently required less outdoor data to train the model. Moreover, it was successful at performing relocalization and pose prediction from the camera images. Although spacecraft attitude estimation must be manipulated using data regarding the spacecraft's position and orientation relative to the camera rather than calculating the pose of the camera itself, the principle is still applicable. Artificial intelligence (AI) studies are concerned with constructing correlations between input and output data. Mahendran et al. [18]  developed a pose estimation algorithm for single objects using a pretrained VGG-M model. Using the Pascal 3D+ dataset, the training process adopted geodesic distance as the loss function. The next year  [19] , used a ResNet-50 model as the base architecture and demonstrated the use of various loss functions such as simple/naïve, log-Euclidean, geodesic, and probabilistic loss on the same dataset (Pascal 3D+).
Another work involving single object detection in  [20]  applied state-of-the-art AI methods to medical science. They implemented CNNs to estimate six degrees of freedom, including the position and attitude of the human brain, from MRI scans. The pose estimation model was constructed using a ResNet18 model, which reduced the required size of the training dataset. In the training stage, the position loss was the mean-squared error, while the orientation loss was the geodesic distance. In addition, some works have addressed multiple-object pose estimation, such as  [21] [22] [23] [24]. The contributions from these works could be applied to multiple-object detection in space. For example, in situations where multiple objects need to interact, such a pose estimation system may need to estimate the poses of the various objects individually.
Various works have used different formulations to address the terms of position loss and orientation loss. For position loss, most of the works implemented mean squared error  [20, 24]  as the loss function. However, some research was successful using the Euclidean distance  [15, 17]  and a multiplication of the scaling coefficient, as shown in Equation  (1). Loss position = β x x i -x gt , (1) where x i is the trial position vector from the layers of the CNN model and x gt is the ground-truth position vector available in the dataset. In many studies  [14, 18, 20] , the orientation loss was formulated as the geodesic loss in Equation (2). Loss orientation = β q arccos q T i q gt , ( where q i is the trial quaternion component extracted by the layers of the CNN model and q gt is the ground-truth quaternion component, which is available in the dataset.
Materials and Methods This research aims to construct an attitude and position estimation model by repurposing a pretrained model. Figure  1  describes the brief information about the construction of the pose estimation model through the training and testing process with the dataset. The details of implementation are described in this section, including dataset preparation, pose estimation algorithm and preprocessing, and construction of the pose estimation algorithm with different loss functions, which consists of both 1) the exponential-based model and 2) the weighted Euclideanbased model.
Dataset Preparation Before repurposing a pretrained model, a dataset must be prepared. The dataset has a critical role in the learning process. A dataset generated solely by the researcher might lead to inaccurate and invalid dataset. For this reason, this experiment collected data from public datasets. Proença and Gao  [15]  provided a publicly available spacecraft dataset suitable for practicing pose estimation with supervised learning algorithms. In the Unreal Engine 4 simulator, the object moving frame was mounted on the spacecraft, and the spacecraft images were acquired by the camera, which was.
Dataset Preparation Before repurposing a pretrained model, a dataset must be prepared. The dataset has a critical role in the learning process. A dataset generated solely by the researcher might lead to inaccurate and invalid dataset. For this reason, this experiment collected data from public datasets. Proença and Gao  [15]  provided a publicly available spacecraft dataset suitable for practicing pose estimation with supervised learning algorithms. In the Unreal Engine 4 simulator, the object moving frame was mounted on the spacecraft, and the spacecraft images were acquired by the camera, which was considered the reference frame. Therefore, the relative parameters of position and attitude were described by the position and attitude of the spacecraft with respect to the camera. In the provided dataset (refer to the example in Figure  2 ), the background image features include the Earth's surface, outer space, and simulated directional light reflections.
For entire image processing, the lighting condition is a significant challenge for the feature detection of convolutional layers. As has been proved by Volpe et al. [25] , there are differences between the reconstructed shape and the CAD model, which indicates that the lighting condition affects the difficulty in feature identification. By capitalizing on the capabilities of Unreal Engine 4, the images dataset was generated from the simulation of realistic scenes. Information about the relative position and relative attitude and the labeled numerical ground-truth data were collected in the training stage and served as the testing reference. Moreover, the transformation values include relative position and relative attitude, which consists of Euler angles, can be converted to quaternions. The attitude conversion object overcomes the problems of singularity and increases the computational performance.
Aerospace 2020, 7, x FOR PEER REVIEW 6 of 21 described by the position and attitude of the spacecraft with respect to the camera. In the provided dataset (refer to the example in Figure  2 ), the background image features include the Earth's surface, outer space, and simulated directional light reflections. For entire image processing, the lighting condition is a significant challenge for the feature detection of convolutional layers. As has been proved by Volpe et al. [25] , there are differences between the reconstructed shape and the CAD model, which indicates that the lighting condition affects the difficulty in feature identification. By capitalizing on the capabilities of Unreal Engine 4, the images dataset was generated from the simulation of realistic scenes. Information about the relative position and relative attitude and the labeled numerical ground-truth data were collected in the training stage and served as the testing reference.
To satisfy GoogLeNet's format, these small images were then center-cropped to 224 × 224 pixels (Figure  5 ). If the images were resized to an even smaller resolution, image details would be lost because the cropping process removes details at the edges of the images. The numerical output for the spacecraft pose is given by a seven-column dataset. In this research, the mathematical pose expression is defined as follows: Poses = x, y, z, q , q , q , q. (4) This seven-dimensional vector must be sliced into two parts: Position = x, y, z , (5) and Attitude = q , q , q , q , where x, y, and z are the magnitude of the relative distance in the X, Y, and Z-axes, respectively, and q represents the real part of the quaternions, while q , q , and q are the components in the vector part. The Soyuz images were used as model input.
Nevertheless, the spacecraft images must be reproduced in a suitable form. This reproduction stage is called image preprocessing. The image format GoogLeNet uses as the input consists of 224 × 224 pixel images. Thus, the original image files were transformed into the smallest possible resolution. First, the original images (which were 1280 × 960 pixels) were resized to be four times smaller (to 320 × 240 pixels). To satisfy GoogLeNet's format, these small images were then center-cropped to 224 × 224 pixels (Figure  5 ). If the images were resized to an even smaller resolution, image details would be lost because the cropping process removes details at the edges of the images. The numerical output for the spacecraft pose is given by a seven-column dataset. In this research, the mathematical pose expression is defined as follows: Poses = x, y, z, q 1 , q 2 , q 3 , q 0.
(8) For the third trial, the total loss value tended to converge to zero. Thus, this function can be taken as the orientation term in Equation (3). Then, the mathematical model of the total loss is defined as follows: Loss total = β x x i -x gt + β q exp 1 -q T i q gt. ( 9 ) The result of this mathematical expression was minimized using the Adam optimizer. In each iteration, the optimizer changed the weights and biases according to the descending gradient during artificial neural model development. The most acceptable values of each scaling coefficient, β x and β q , can be found by trial and error. The values of these scaling coefficients are dataset dependent; therefore, models trained on different datasets may have different scaling coefficients.
Results of Weighted Euclidean-Based Pose Estimation Model The final model construction was based on a modified version of GoogLeNet and implemented in a Python environment with 4500 generated images in the training dataset of the Soyuz spacecraft. The model was trained for 30,000 iterations using Equation (  22 ) as the loss function and optimized by the Adam optimizer. The model was trained on the GPUs of the compute nodes at the National Astronomical Research Institute (NARIT). The experiment on weighted Euclidean loss function was studied by evaluating the differences in the learning behavior and the performance of each model under different orientation scaling coefficients. The learning behavior is considered as the history of the iteration loss during the training process. Figure  8a  shows the overall learning behavior of each model with different orientation scaling coefficients, while the illustration in Figure  8b  focuses on the events after the loss was reduced to less than 2500 to more closely inspect the learning behavior.