dataesr/scientific-paragraphs-categorization huggingface dataset:
The regressed queries of detected objects will be added to the tracking queries in the next time step. Subsequently, the decoder will process all detection and tracking queries auto-regressively. Hence, jointly-detection-and-tracking is done within the system using an extended query mechanism. We evaluate our model with the state-of-the-art Transformer-based models on the MOT17 dataset during the experiment. Our model achieves prominent results compared with the Transformer-based competitors while the model size shrunk by up to 30. 7%. Ablation research is conducted to show the effectiveness of our new architecture. In this way, we hope the researchers can go further in this direction with the unified tracking model and exploit more Transformers and green learning for CV. In summary, we listed our contributions as follows: • A modified open-sourced 1 Transformer backbone, which can be a substitution of convolutional backbones in any CV tasks for arbitrary image sizes.
Tracking-by-detection These approaches mainly focus on the tracking task using existing detections from other detectors. In the early stages, works like  Sanchez-Matilla et al. (2016)  and  Bewley et al. (2016)  solely rely on the detection boxes and their probabilities. The tracking is achieved by data association using a particle filter framework or a bipartite matching between detected boxes and the estimated boxes. They perform moderately but suffer from low accuracy and identity switches because of the density of objects and the occlusions. The evolution of machine learning and the involvement of image features in MOT systems remarkably boosted tracking performance. As many robust object detectors emerge (Ren et al. , 2015; Duan et al. ,   1 GitHub link: https://github. com/simonwu53/MOTT.
, 2021) , a cornerstone of the Transformer detector. The TransTrack duplicates the decoder of deformable DETR, utilizing one for detection and another for tracking. This method is straightforward without additional data association. However, the backbone and the second decoder make it an enormous model. Meinhardt et al. (2022) ,  Zeng et al. (2022)  and Xu and Vivet (2021) improved a step further by tracking-by-query mechanism, which only involves the backbone and a full deformable DETR in this paradigm. Object queries are categorized into detection queries and tracking queries. The deformable decoder will decode all types of queries at once. Hence, detection and tracking processes happen concurrently. Despite the simplification of the model, they are still a combination of two complete models.
Methodology It is known to all that a full encoder-decoder-based Transformer is good at sequence prediction problems by the interaction between object queries and memory embedding. We formulate MOT as a sequence prediction as other Transformer-based MOT systems. Based on the characteristics of different types of Transformers, we create a customized encoder-decoder architecture specialized in MOT, namely MOTT, as shown in Fig. 2. The model adheres to a general encoderdecoder Transformer architecture, where the encoder also functions as a backbone to extract features and encode information simultaneously. At every time step, an image 𝐼 is fed into the model for feature extraction and encoding. The output of the encoder will be queried by learnable object queries comprised of fix-sized detection queries 𝑞 𝑑 and tracking queries 𝑞 𝑡𝑟 in the decoder.
Transformer encoder In this work, we leverage a novel Transformer encoder called CSWin  (Dong et al. , 2022) , a general-purpose vision backbone. We select it for three reasons: it has a better performance compared with conventional ResNet backbone; it is a Transformer backbone, also an encoder, which significantly diminishes the model parameters and computations compared with the mixture of a backbone and an encoder; it is fully open-sourced. The CSWin model, as shown in Fig. 3 , is an encoder-only model with four stages like ResNet, where each stage has a different amount of encoder layers 𝑁 𝑖 , and the size of output feature maps will be shrunk by two after every stage. Within each layer is a cross-shaped window selfattention (CSWin-SA) module, dividing the feature maps into two heads by depth (as shown in Fig.
4 ). A dynamic stripe window 𝑠𝑤 divides the feature map into smaller stripes vertically and horizontally, followed by the CSWin-SA applied on each stripe in parallel. Hence, the module only attends to the local context within the stripes to extract features. 𝑠𝑤 is wider as the stage goes deeper because the receptive field is larger. Later, these features will be merged into global features using a linear layer. However, this model is limited to specific datasets like ImageNet  (Deng et al. , 2009)  and COCO  (Lin et al. , 2014)  because of the split size 𝑠𝑤 in the CSWin-SA module. The module will not work when the sizes of feature maps cannot be divided by 𝑠𝑤 evenly. This constraint remarkably hinders the downstream CV tasks. Thus, we propose a padding mechanism to make it work with arbitrary images as shown in Fig.
4. For an arbitrary feature map 𝑓 𝑐 𝑖 ×ℎ 𝑖 ×𝑤 𝑖 𝑖 of stage 𝑖 ∈ [1, 4] and the 𝑠𝑤 𝑖 of stage 𝑖, if any side 𝑙 of the feature map cannot be divided by 𝑠𝑤 𝑖 evenly (e. g. ℎ 𝑖 mod 𝑠𝑤 𝑖 ≠ 0), we add zero-paddings to that side using Eq. (  2 ) before the CSWin-SA module. 𝑙 𝑝𝑎𝑑 = 𝑠𝑤 𝑖 -(𝑙 mod 𝑠𝑤 𝑖 ) ∀𝑙 ∈ {ℎ 𝑖 , 𝑤 𝑖 } ∧ 𝑙 mod 𝑠𝑤 𝑖 ≠ 0 𝑙 𝑎 = 𝑙 𝑝𝑎𝑑 | 2 𝑙 𝑏 = 𝑙 𝑝𝑎𝑑 -𝑙 𝑎 (2) where 𝑙 𝑝𝑎𝑑 is the total padding size for both extremities of a side 𝑙. 𝑙 𝑎 and 𝑙 𝑏 are the padding sizes of each extremity. Local feature extraction is convolutional networks' specialty as they can easily extract features like eyes, noses, and ears by limiting their receptive field. This differs for the Transformers because the attention mechanism focuses on the whole input. CSWin-SA controls its receptive field by dynamic 𝑠𝑤 strengthening local feature extraction while drawing the global features from other heads.
Transformer decoder Before the interaction with queries in the decoder, multi-level feature maps need to be preprocessed. As shown in Eq. (  4 ), feature maps will be projected into 𝑑 ℎ channels to match the queries. Next, two spatial dimensions of the feature maps will be merged, followed by a concatenation of all flattened features generating a memory embedding 𝜒 𝑒𝑛𝑐 for the decoder. 𝜒 𝑝 = {𝑓 𝑐 𝑖 ×ℎ 𝑖 ×𝑤 𝑖 𝑖 → 𝑓 𝑑 ℎ ×ℎ 𝑖 ×𝑤 𝑖 𝑖 | ∀𝑓 𝑖 ∈ 𝜒 𝑒 } 𝜒 𝑓 = {𝑓 𝑑 ℎ ×ℎ 𝑖 ×𝑤 𝑖 𝑖 → 𝑓 𝑑 ℎ ×ℎ𝑤 𝑖 𝑖 | ∀𝑓 𝑖 ∈ 𝜒 𝑝 } 𝜒 𝑒𝑛𝑐 = Concat(𝜒 𝑓 ) (4) Decoders are intended for performing detection and tracking simultaneously using object queries and memory embedding. In order to decode massive queries while keeping the computational cost low, we only select the decoder from the deformable DETR  (Zhu et al. , 2021). This decoder can regress the bounding boxes using its efficient Multiscale Deformable Attention module (MSDAttn), whose computational complexity is irrelevant to the input size.
Post process For every query from the decoder, the bounding box is predicted by a three-layer MLP module, while the class label is generated from a fully-connected layer directly. Because the MSDAttn utilizes reference points for attention, the predicted bounding boxes are the offsets relative to the center of reference points. In this implementation, 𝑁 𝑑𝑒𝑐 cloned MLP modules are created for iterative bounding box refinement, where 𝑁 𝑑𝑒𝑐 is the number of decoder layers. These MLP modules will be injected into the decoders to update the coordinates of reference points on every decoder layer.
Dataset We conducted our experiments on the MOT17 dataset  (Milan et al. , 2016)  provided by MOTChallenge Benchmark to compare with other competitors. It is a multi-object tracking dataset favored by most of the works we researched. The dataset contains 14 videos in total, where seven are for training and the remaining seven are for testing. Noted that there are only annotations for training sequences. In order to get evaluation results on testing sequences, one must create a new entry on their benchmark and submit the tracking results to their server to get the scores. Many works include additional training data and tuning to get a competitive score. Hence, in our experiments, we use data differently in order to control variables: • Benchmark: All training data, including additional ones (mention below).
In addition to data splits, MOTChallenge also provides two tracks for competition: public and private detections. Methods based on public detection only associate spatiotemporal tracklets using the detections provided in the dataset. Private-detection methods perform detection and tracking by themselves. Hence, our method is an online private one that performs one-shot prediction for every incoming frame. As mentioned before, CrowdHuman dataset  (Shao et al. , 2018 ) is another dataset we used for benchmarking. Because the MOT17 dataset is relatively small, CrowdHuman could be a great complement. CrowdHuman dataset focuses on human detection, which has 15 000, 4370, and 5000 images for training, validation, and testing, respectively. Although it is not a sequence-based dataset, we can imitate two consecutive frames by cropping and shifting the original image.
Implementation details We trained several models for a comprehensive comparison. Firstly, we trained vanilla TrackFormer  (Meinhardt et al. , 2022)  and TransTrack  (Sun et al. , 2020)  as two online private competitors. Secondly, we substitute the ResNet backbone in TrackFormer with our modified CSWin model to test the ability of the state-of-the-art Transformer backbone (denoted as TrackFormer-CSWin). Thirdly, our proposed efficient model MOTT. In this paper, we use the tiny variant of CSWin in all corresponding models, which features 25 layers with an initial embedding size of 64 and produces a three-level feature pyramid from stages 2, 3, and 4. The encoder is pre-trained with COCO dataset  (Lin et al. , 2014). For the decoder, we leverage deformable DETR's decoder, featuring 6 layers with an embedding size of 288 and 8 attention heads.
It takes a fourlevel feature pyramid, where the last level is derived from the result of the last stage of the CSWin model by a convolutional layer. Besides, we set the number of detection queries to 500. The total number of object queries varies based on how many objects are tracked among frames. For the benchmark model, the model is pre-trained on the Crowd-Human dataset for 80 epochs. Then, we train the model on a mixed dataset combining MOT17 and CrowdHuman for 40 epochs. For the comparison models, we pre-train the models as before. Next, all models are trained on the first half of the MOT17 training set for 40 epochs and evaluated on the second half. During all training, we use AdamW optimizer  (Loshchilov and Hutter, 2019)  with the initial learning rate at 2 × 10 -4. The learning rate drops by a factor of 10 at 50 epochs for the pretraining stage; Then, it drops again at 10 epochs for the second stage.
Computing efficiency As a backbone, CSWin (the tiny variant we adopted) requires slightly more floating point operations per second (FLOPS) with 4. 3 × 10 9 compared to ResNet50 with 3. 8 × 10 9 used in all competitors on the ImageNet dataset. Nonetheless, it surpasses ResNet50 in object detection and segmentation tasks by a large margin on the COCO dataset in the experiment conducted by  Dong et al. (2022). It is known to all that the computation cost of the attention mechanism is extremely high in the image domain because of the highresolution images  (Zhu et al. , 2021). The computational complexity of multi-head attention is 𝑂(𝑁 𝑞 𝐶 2 + 𝑁 𝑘 𝐶 2 + 𝑁 𝑞 𝑁 𝑘 𝐶), where 𝑁 𝑞 , 𝑁 𝑘 , and 𝐶 are the length of (object) query, key (flattened feature map), and depth (channel), respectively. In the DETR encoder  (Carion et al.
Ablation study In this section, we reveal the effects of each module in MOTT along with the TrackFormer, which is our baseline model. Table  3  presents the performance impact of each module. The base model consists of a ResNet-50 backbone, a deformable encoder, and a deformable decoder. When we change only the backbone to CSWin, the model achieves a considerable performance boost from 66. 8% to 72. 7% in MOTA and from 70. 7% to 72. 9% in IDF1, which shows the capability of the novel Transformer-based backbone. However, CSWin is more computationally-intensive than ResNet, as we previously reviewed, making the system even slower. Since CSWin and the deformable encoder are pursuing the same objective, we remove the deformable encoder entirely and get on par performance at 71. 9% in MOTA and 72.
6% in IDF1. Furthermore, the speed of the final model almost doubled. The reason we remove the deformable encoder module is that it is not efficient. We can barely get any improvements from such a big module. Tracking happens inside the decoder by transforming object queries into embeddings holding information from tracked objects. The key, or flattened feature map, is required by object queries in a decoder. However, we cannot use the feature map from the ResNet backbone directly because it has to be encoded. MOTT is different since our backbone is an encoder itself. Thus, we choose to remove the deformable encoder. The performance shows that CSWin is capable of extracting features as well as encoding by the cross-shaped self-attention mechanism. Additionally, the deformable encoder still puts attention over the entire image, demonstrating a deficiency in local attention.
Evaluation on different datasets Firstly, we run the benchmark on the MOT17 testing set with our MOTT following the training steps in 4. 3. The results from the MOTChallenge leaderboard are reported in Table  2. Our method takes a single image at each time step, and outputs detected and tracked objects. Thus, it is categorized as a private online method in the table. MOTT achieves competitive results with the state-of-the-art methods. The highest score in MOTA means an overall better capability in MOT. MT and ML show the ability to track objects among frames. Compared to the second-best results, our method gets 4. 9% more in MT and 9. 1% less in ML. FN is another essential criterion because catastrophic situations may happen if the system misses an object in an unmanned scenario. Our model keeps a low FN result by providing much more detections, which also contributes to the occasions of the identity switches.
Another reason for frequently making identity switches is that it cannot memorize an inactive object long-term with the current simple tracking query mechanism. In   decoder is also efficient and great at decoding the queries based on selected key embedding. To be mentioned, the proposed model is much lighter than the others considering the number of parameters and FLOPS. Next, we evaluated MOTT on the testing sets of MOT20 and Dance-Track in addition to MOT17. Table  5  unveils more insights into our model from different perspectives. The MOT17 and MOT20 results are obtained from the model trained by corresponding datasets, while the DanceTrack results are derived from the MOT20 model without  achieved by the MOT20 model, which means MOTT can be generalized in various use cases. Finally, we show the model outputs in Fig.
5 , corroborating the statistics in Table  5. In MOT17, the model manages to track most people but has low confidence in riding people due to insufficient training data in this case. MOT20 shows the capacity of MOTT. Because of NMS and occlusion, few bounding boxes among the crowd are eliminated. Furthermore, MOTT detects people in various poses, demonstrating invariance to translation, rotation, and scale by learning from the MOT20 dataset.
Conclusion In this paper, we proposed a new Transformer-based MOT architecture, namely MOTT, which could save much on the hardware cost and energy while retaining the state-of-the-art MOT performance. By only leveraging the effective modules based on its specification, the new model only contains an encoder and a decoder for the challenging MOT task. Our model achieves a competitive score in MOTA at 73. 2% with up to 62% fewer FLOPS than a typical Transformer-based MOT model. The model shows the potential of moving towards a green learning paradigm in CV tasks. In future work, we will focus on improving the object queries to solve the issues with re-identification. We hope this new architecture could enlighten others on the balance of performance and computational cost.
Perturbations of the oxidative stress module affect response of the iron module but the converse is not true To gain a better understanding of how the oxidative stress and the iron homeostasis modules interact and influence Fe-S cluster biogenesis, we simulated simple or combination of KO mutations affecting one or several modules. Such single or multiple mutants can be easily defined with the software GINsim (Materials and Methods). We ran simulations of KO of the OxyR node (affecting the oxidative stress module), of the Fur node (affecting the iron homeostasis module), and of both. Fig. 4  shows the asymptotical behaviors of H 2 O 2 and Fe free nodes in the WT strain for these three perturbated situations. Expectedly, mutations affect the module to which the node belongs by blocking the oscillations.
Model predicts growth defects situations Deficiencies in Fe-S cluster biogenesis are known to cause growth defects. We thus used the model to predict KO mutants for which the Isc and Suf nodes would be turned off at particular input O 2 and Fe ext values. The prediction is that such mutants should exhibit growth defects in these environmental conditions as Fe-S cluster biogenesis cannot be fully ensured. Out of 54 simple mutants, only the IscR-A KO and the Suf KO mutants displayed situations in which both Isc and Suf nodes equaled 0. For combinations of double mutants, only combinations of mutants including IscR-A KO or Suf KO mutation displayed situations for which nodes Isc and Suf were both turned down to 0. The ensemble of these data is available online (https://gitlab. com/Laurent. Tichit/fe-s-cluster -biogenesis-logical-model/tree/master/Results/S4_Fig).
Logical modeling Software GINsim We used the freely available software GINsim (Gene Interaction Network simulation, http://ginsim. org) for the edition, analysis and simulation of the regulatory graph  (Chaouiya et al. 2012 ). Once the model defined, several simulation tools are proposed to simulate the State Transition Graph, the Hierarchical Transition Graph, do reduction of model or compute the stable states. GINsim also enables the simulation of mutants (loss-of-function, ectopic gene expression, and combinations) by blocking the levels of expression of the corresponding variables. To further ease the analysis of multiple perturbations, we have written a set of scripts in Python and R, which iteratively compute the behavior of our model for all input values and mutants considered, and generate heatmaps representing their behavior.
, 2015. All simulations were run from 1965 to 2014 (with a time step of 1 day) using FEFLOW 7. 5. We use trial and error to settle for the final set of parameters because a) FEFLOW 7. 5 parameter estimation program (FEPEST) is highly time consuming and buggy, and b) 180 Dirichlet BCs are variables and cannot be optimized as parameters using FEPEST. FEFLOW 7. 5 produces a hydraulic head field but does not explicitly display discharge. Although it shows nodal and elemental Darcy flux, it does not provide a time series of Darcy discharge across individual elements, so we cannot add up Darcy flux across elements to get a net discharge time series. Furthermore, whether the net discharge will match the discharge of ETNA will largely depend on the method used to calculate the discharge. Thus, the only way to compare the water release rates of 185 the fast and the slow stores of ETNA and FEFLOW is to compare their discharge recessions.
Funder 1 : 1 ANR Ref: ATCHA ANR-16-CE03-0006. Funder 2: Indian Institute of Science Ref: SP/INRA-17-0002 Code/Data availability: All data sets are publicly available in the agrhys repository of INRAE, France. Link to data: Grapheur de VIDAE (agrhys. fr) 425 Link to code and user instructions provided in supplement. https://doi. org/10. 5194/hess-2023-112 Preprint. Discussion started: 30 May 2023 c Author(s) 2023. CC BY 4. 0 License.
Those models speed convergence by reducing function evaluation cost and/or smoothing noisy response functions  (Hemker et al. , 2008). Magnier and Haghighat (2010)  used TRNSYS simulations to train an artificial neural network (ANN), then used the trained ANN to couple with a genetic algorithm (GA) to optimize the thermal comfort and the energy consumption. In their study, the database for training the ANN consisted of output of 450 simulations. It took 3 weeks to generate the database, but optimization time was very small. If direct coupling between TRNSYS and GA was used, it would need 10 years to finish the task according to  Magnier and Haghighat (2010). Other examples include the studies done by  Chen et al. (2010)  who used a feed forwards neural network for the identification of temperature in intelligent buildings and then applied optimization by particle swarm optimization (PSO).
(2021)  who performed optimization of the structural and architectural parameters of a detached passive building located in Serbia with a sunspace using a non-dominant sorting genetic algorithm (NSGA-II). The optimization was run using DesignBuilder software package coupled to EnergyPlus™ dynamic building energy simulation software. The defined variable optimization parameters include window-to-wall ratio (WWR) of each individual building façade, type of glazing, different façade wall constructions, and window shading system on the façades.
Application of the sequential multi-objective optimization methodology A 12-day sequence obtained from a single tested individual of the building model and three selection criteria (backup energy, energy stored in the tank and internal room temperature) was used. The values of the parameters were fixed as: SCOLL = 9. 28 m 2 , VST = 0. 763 m 3 and INS = 0. 12 m. For NSGA-II, the same model performances (investment cost and annual need of backup energy) were evaluated for comparison with the Pareto front obtained upon an annual simulation. Fig. 5  shows the Pareto front obtained (in blue) with respect to the reference Pareto front (in black). Firstly, the optimization algorithm converged much faster than the reference one. It took only 5. 7 h to converge when using the reduced sequence while it took 175 h when using the annual data profiles.
SVRAI NDS with cars The first data used in this study come from the SVRAI project (Saving Lives through Road Incident Analysis Feedback). Specific Event Data Recorders (EDRs) implemented in 46 light vehicles of five public fleets located in three different regions of France acquired data during 12 months (2012-2013, see Fig. 1 ). All were company vehicles dedicated or not to one person. 221 drivers volunteered to take part in the experiment and could potentially drive the cars. The device described by  Lechner and Naude (2011)  was designed for the purpose of detecting some critical driving situations, called incidents, in which a vehicle faces high dynamics demands in the longitudinal, lateral or combined directions, as well as potential accidents. It is fixed under the driver seat and continuously acquires analog data from sensors directly integrated into the EDR (3 axis accelerometers and gyro meters for naturalistic driving, and 2 axis accelerometers for crashes), data provided by a GPS (trajectory and speed), and data available on the vehicle on-board diagnosis system (CAN bus, information depending on the car model).