Miracle-dz/UniFAR-FTUs huggingface dataset:
Adaptive Gradient Prediction for DNN Training

Neural network training is inherently sequential where the layers ﬁnish the forward propagation in succession, followed by the calculation and back-propagation of gradients (based on a loss function) starting from the last layer. The sequential computations signiﬁcantly slow down neural network training, especially the deeper ones. Prediction has been successfully used in many areas of computer architecture to speed up sequential processing. Therefore, we propose ADA-GP, that uses gradient prediction adaptively to speed up deep neural network (DNN) training while maintaining accuracy. ADA-GP works by incorporating a small neural network to predict gradients for different layers of a DNN model. ADA-GP uses a novel tensor reorganization to make it feasible to predict a large number of gradients. ADA-GP alternates between DNN training using backpropagated gradients and DNN training using predicted gradients. ADA-GP adaptively adjusts when and for how long gradient prediction is used to strike a balance between accuracy and performance. Last but not least, we provide a detailed hardware extension in a typical DNN accelerator to realize the speed up potential from gradient prediction. Our extensive experiments with fourteen DNN models show that ADA-GP can achieve an average speed up of 1 . 47 × with similar or even higher accuracy than the baseline models. Moreover, it consumes, on average, 34% less energy due to reduced off-chip memory accesses compared to the baseline hardware accelerator.
CIVA-Stabilized Finite Element Method for Tsunami Simulations

This paper presents a numerical method using the CIVA-stabilized finite element method based on SUPG (CIVA-SUPG) for tsunami simulations. The Boussinesq equation is employed for the governing equation in order to treat both the wave nonlinearity and dispersion effects. The equation is divided into two phases, an advection phase and a non-advection phase. The CIVA method is employed to the advection phase and the stabilized finite element method based on SUPG is employed to the discretization for non-advection phase. The present method is applied to several benchmark examples to show the validity and efficiency of the method. For the application example, the present method is employed to the tsunami simulation generated by Great East-Japan earthquake.
An Approach to Multiple Comparison Benchmark Evaluations that is Stable Under Manipulation of the Comparate Set

The measurement of progress using benchmarks evaluations is ubiquitous in computer science and machine learning. However, common approaches to analyzing and presenting the results of benchmark comparisons of multiple algorithms over multiple datasets, such as the critical difference diagram introduced by Dem\v{s}ar (2006), have important shortcomings and, we show, are open to both inadvertent and intentional manipulation. To address these issues, we propose a new approach to presenting the results of benchmark comparisons, the Multiple Comparison Matrix (MCM), that prioritizes pairwise comparisons and precludes the means of manipulating experimental results in existing approaches. MCM can be used to show the results of an all-pairs comparison, or to show the results of a comparison between one or more selected algorithms and the state of the art. MCM is implemented in Python and is publicly available.
Lifting the Curse of Capacity Gap in Distilling Language Models

Pretrained language models (LMs) have shown compelling performance on various downstream tasks, but unfortunately they require a tremendous amount of inference compute. Knowledge distillation finds a path to compress LMs to small ones with a teacher-student paradigm. However, when the capacity gap between the teacher and the student is large, a curse of capacity gap appears, invoking a deficiency in distilling LMs. While a few studies have been carried out to fill the gap, the curse is not yet well tackled. In this paper, we aim at lifting the curse of capacity gap via enlarging the capacity of the student without notably increasing the inference compute. Largely motivated by sparse activation regime of mixture of experts (MoE), we propose a mixture of minimal experts (MiniMoE), which imposes extra parameters to the student but introduces almost no additional inference compute. Experimental results on GLUE and CoNLL demonstrate the curse of capacity gap is lifted by the magic of MiniMoE to a large extent. MiniMoE also achieves the state-of-the-art performance at small FLOPs compared with a range of competitive baselines. With a compression rate as much as ~50\times, MiniMoE preserves ~95% GLUE score of the teacher.
U-TILISE: A Sequence-to-Sequence Model for Cloud Removal in Optical Satellite Time Series

Satellite image time series in the optical and infrared spectrum suffer from frequent data gaps due to cloud cover, cloud shadows, and temporary sensor outages. It has been a long-standing problem of remote sensing research how to best reconstruct the missing pixel values and obtain complete, cloud-free image sequences. We approach that problem from the perspective of representation learning and develop U-TILISE, an efficient neural model that is able to implicitly capture spatio-temporal patterns of the spectral intensities, and that can therefore be trained to map a cloud-masked input sequence to a cloud-free output sequence. The model consists of a convolutional spatial encoder that maps each individual frame of the input sequence to a latent encoding; an attention-based temporal encoder that captures dependencies between those per-frame encodings and lets them exchange information along the time dimension; and a convolutional spatial decoder that decodes the latent embeddings back into multi-spectral images. We experimentally evaluate the proposed model on EarthNet2021, a dataset of Sentinel-2 time series acquired all over Europe, and demonstrate its superior ability to reconstruct the missing pixels. Compared to a standard interpolation baseline, it increases the PSNR by 1.8 dB at previously seen locations and by 1.3 dB at unseen locations.
Non-flat ABA is an Instance of Bipolar Argumentation

Assumption-based Argumentation (ABA) is a well-known structured argumentation formalism, whereby arguments and attacks between them are drawn from rules, defeasible assumptions and their contraries. A common restriction imposed on ABA frameworks (ABAFs) is that they are flat, i.e. each of the defeasible assumptions can only be assumed, but not derived. While it is known that flat ABAFs can be translated into abstract argumentation frameworks (AFs) as proposed by Dung, no translation exists from general, possibly non-flat ABAFs into any kind of abstract argumentation formalism. In this paper, we close this gap and show that bipolar AFs (BAFs) can instantiate general ABAFs. To this end we develop suitable, novel BAF semantics which borrow from the notion of deductive support. We investigate basic properties of our BAFs, including computational complexity, and prove the desired relation to ABAFs under several semantics.
Mapping Biological Neuron Dynamics into an Interpretable Two-layer Artificial Neural Network

Dendrites are crucial structures for computation of an individual neuron. It has been shown that the dynamics of a biological neuron with dendrites can be approximated by artificial neural networks (ANN) with deep structure. However, it remains unclear whether a neuron can be further captured by a simple, biologically plausible ANN. In this work, we develop a two-layer ANN, named as dendritic bilinear neural network (DBNN), to accurately predict both the sub-threshold voltage and spike time at the soma of biological neuron models with dendritic structure. Our DBNN is found to be interpretable and well captures the dendritic integration process of biological neurons including a bilinear rule revealed in previous works. In addition, we show DBNN is capable of performing diverse tasks including direction selectivity, coincidence detection, and image classification. Our work proposes a biologically interpretable ANN that characterizes the computation of biological neurons, which can be potentially implemented in the deep learning framework to improve computational ability.
OntoType: Ontology-Guided Zero-Shot Fine-Grained Entity Typing with Weak Supervision from Pre-Trained Language Models

Fine-grained entity typing (FET), which assigns entities in text with context-sensitive, fine-grained semantic types, will play an important role in natural language understanding. A supervised FET method, which typically relies on human-annotated corpora for training, is costly and difficult to scale. Recent studies leverage pre-trained language models (PLMs) to generate rich and context-aware weak supervision for FET. However, a PLM may still generate a mixture of rough and fine-grained types, or tokens unsuitable for typing. In this study, we vision that an ontology provides a semantics-rich, hierarchical structure, which will help select the best results generated by multiple PLM models and head words. Specifically, we propose a novel zero-shot, ontology-guided FET method, OntoType, which follows a type ontological structure, from coarse to fine, ensembles multiple PLM prompting results to generate a set of type candidates, and refines its type resolution, under the local context with a natural language inference model. Our experiments on the Ontonotes, FIGER, and NYT datasets using their associated ontological structures demonstrate that our method outperforms the state-of-the-art zero-shot fine-grained entity typing methods. Our error analysis shows that refinement of the existing ontology structures will further improve fine-grained entity typing.
Decomposed Prompting for Machine Translation Between Related Languages using Large Language Models

This study investigates machine translation between related languages i.e., languages within the same family that share linguistic characteristics such as word order and lexical similarity. Machine translation through few-shot prompting leverages a small set of translation pair examples to generate translations for test sentences. This procedure requires the model to learn how to generate translations while simultaneously ensuring that token ordering is maintained to produce a fluent and accurate translation. We propose that for related languages, the task of machine translation can be simplified by leveraging the monotonic alignment characteristic of such languages. We introduce DecoMT, a novel approach of few-shot prompting that decomposes the translation process into a sequence of word chunk translations. Through automatic and human evaluation conducted on multiple related language pairs across various language families, we demonstrate that our proposed approach of decomposed prompting surpasses multiple established few-shot baseline approaches. For example, DecoMT outperforms the strong few-shot prompting BLOOM model with an average improvement of 8 chrF++ scores across the examined languages.
Accelerating Graph Neural Networks via Edge Pruning for Power Allocation in Wireless Networks

Graph Neural Networks (GNNs) have recently emerged as a promising approach to tackling power allocation problems in wireless networks. Since unpaired transmitters and receivers are often spatially distant, the distance-based threshold is proposed to reduce the computation time by excluding or including the channel state information in GNNs. In this paper, we are the first to intro-duce a neighbour-based threshold approach to GNNs to reduce the time complexity. Furthermore, we conduct a comprehensive analysis of both distance-based and neighbour-based thresholds and provide recommendations for selecting the appropriate value in different communication channel scenarios. We design the corresponding neighbour-based Graph Neural Networks (N-GNN) with the aim of allocating transmit powers to maximise the network throughput. Our results show that our proposed N-GNN offer significant advan-tages in terms of reducing time complexity while preserving strong performance and generalisation capacity. Besides, we show that by choosing a suitable threshold, the time complexity is reduced from $\mathcal{O}(\vert \mathcal{V}\vert ^{2})$ to $\mathcal{O}(\vert \mathcal{V}\vert)$, where $\vert \mathcal{V}\vert$ is the total number of transceiver pairs.
Enhancing Coherence of Extractive Summarization with Multitask Learning

This study proposes a multitask learning architecture for extractive summarization with coherence boosting. The architecture contains an extractive summarizer and coherent discriminator module. The coherent discriminator is trained online on the sentence vectors of the augmented textual input, thus improving its general ability of judging whether the input sentences are coherent. Meanwhile, we maximize the coherent scores from the coherent discriminator by updating the parameters of the summarizer. To make the extractive sentences trainable in a differentiable manner, we introduce two strategies, including pre-trained converting model (model-based) and converting matrix (MAT-based) that merge sentence representations. Experiments show that our proposed method significantly improves the proportion of consecutive sentences in the extracted summaries based on their positions in the original article (i.e., automatic sentence-level coherence metric), while the goodness in terms of other automatic metrics (i.e., Rouge scores and BertScores) are preserved. Human evaluation also evidences the improvement of coherence and consistency of the extracted summaries given by our method.
Improving the estimation of the environment parameters via a two-qubit scheme

We demonstrate how using two qubits can drastically improve the estimation of environment parameters as compared to using only a single qubit. The two qubits are coupled to a common harmonic oscillatorenvironment, and the properties of the environment are imprinted upon the dynamics of the two qubits. The reduced density matrix of only one of these qubits contains a decoherence factor as well as an additional factor taking into account the indirect interaction induced between the qubits due to the interaction with their common environment. This additional factor can drastically improve the estimation of the environment parameters, as quantified by the quantum Fisher information. In particular, we investigate the estimation of the cutoff frequency, the coupling strength, and the temperature using our two-qubit scheme as compared to simply using a single qubit. For super-Ohmic environments in particular, one can improve the precision of the estimates by orders of magnitude.
A Reverse ES (CVaR) Optimization Formula

The celebrated Expected Shortfall (ES) optimization formula implies that ES at a fixed probability level is the minimum of a linear real function plus a scaled mean excess function. We establish a reverse ES optimization formula, which says that a mean excess function at any fixed threshold is the maximum of an ES curve minus a linear function. Despite being a simple result, this formula reveals elegant symmetries between the mean excess function and the ES curve, as well as their optimizers. The reverse ES optimization formula is closely related to the Fenchel-Legendre transforms, and our formulas are generalized from ES to optimized certainty equivalents, a popular class of convex risk measures. We analyze worst-case values of the mean excess function under two popular settings of model uncertainty to illustrate the usefulness of the reverse ES optimization formula, and this is further demonstrated with an application using insurance datasets.
Metric of image quality based on structural similarity

A new evaluation method of image quality based on its construction simulation is proposed to solve the limit of evaluation of perceptual distortion by analyzing the current characteristic of image quality evaluation method. Whole similarity obtained from luminance, contrast and image construction is the objective evaluation standard of image quality. The method fully considers the characteristic of structure information of image and vision of people, starts from the comprehension function of image context, and sets up the structure simulation computing model to evaluate the subjective perception to image quality. By theory deducing and algorithm validation, the evidences for selecting the image compressed algorithm and evaluating image quality are obtained. Reconstructed image after encoding by compression algorithm SPIHT (Set Partitioning in Hierarchical Trees) is compared with the traditional evaluation image based on Peak Signal-to-noise Ratio (PSNR), and experiment shows that the method proposed in the paper is a more effective evaluating method for image quality.
A New Feature Selection Algorithm for SVM-Based Fully Polarimetric SAR Image Classification

SVM is used for fully polarimetric SAR image classification,and a novel feature selection algorithm followed by SVM- based classification is proposed in this paper.In the new algorithm,number of support vectors is taken as estimation rule,and sequential forward selection is used.Using L-band fully polarimetric SAR data of Flevoland,Netherlands,acquired by the NASA/JPL AIRSAR sen- sor,the new feature selection algorithm is compared with RELIEF-F,higher classification accuracy with less or equivalent number of fea- tures in a wider range of the SVM parameters is observed from the experiment results.
Lorentz violation in γ-pair production (e+e- → γγ)

In this paper, we investigate the production process of a photon pair in a regime of Quantum Electrodynamics (QED), considering the presence of a constant four-vector that defines a privileged direction in spacetime, resulting in Lorentz symmetry violation and CPT violation. We explore Lorentz violation in the context of vectorial nonminimal coupling, where we analyze the modifications introduced by this coupling in the scattering amplitude at the vertex. Moreover, we delve into Lorentz violation with axial-like nonminimal coupling. Both analyses are conducted by examining the constant four-vector of both time-like and space-like types.
He I temperatures in planetary nebulae

Abstract It is well established that PNe are not thermally homogeneous, beyond what chemically homogeneous photoionization models predict. To discriminate between the different scenarios proposed in the literature, we have determined the temperature based on He I lines for a dozen PNe. We compare these determinations with those derived from the Balmer continua. We find that the He I temperatures agree with the Balmer temperatures and that both are lower than those derived from CELs; suggesting that most PNe are chemically homogeneous.
ZS-MSTM: Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding

In this study, we address the importance of modeling behavior style in virtual agents for personalized human-agent interaction. We propose a machine learning approach to synthesize gestures, driven by prosodic features and text, in the style of different speakers, even those unseen during training. Our model incorporates zero-shot multimodal style transfer using multimodal data from the PATS database, which contains videos of diverse speakers. We recognize style as a pervasive element during speech, influencing the expressivity of communicative behaviors, while content is conveyed through multimodal signals and text. By disentangling content and style, we directly infer the style embedding, even for speakers not included in the training phase, without the need for additional training or fine-tuning. Objective and subjective evaluations are conducted to validate our approach and compare it against two baseline methods.
Using an Observation Protocol To Evaluate Student Argumentation Skills in Introductory Biology Laboratories

Argumentation is vital in the development of scientific knowledge, and students who can argue from evidence and support their claims develop a deeper understanding of science. In this study, the Argument-Driven Inquiry instruction model was implemented in a two-semester sequence of introductory biology laboratories. ABSTRACT Argumentation is vital in the development of scientific knowledge, and students who can argue from evidence and support their claims develop a deeper understanding of science. In this study, the Argument-Driven Inquiry instruction model was implemented in a two-semester sequence of introductory biology laboratories. Student’s scientific argumentation sessions were video recorded and analyzed using the Assessment of Scientific Argumentation in the Classroom observation protocol. This protocol separates argumentation into three subcategories: cognitive (how the group develops understanding), epistemic (how consistent the group’s process is with the culture of science), and social (how the group members interact with each other). We asked whether students are equally skilled in all subcategories of argumentation and how students’ argumentation skills differ based on lab exercise and course. Students scored significantly higher on the social than the cognitive and epistemic subcategories of argumentation. Total argumentation scores were significantly different between the two focal investigations in Biology Laboratory I but not between the two focal investigations in Biology Laboratory II. Therefore, student argumentation skills were not consistent across content; the design of the lab exercises and their implementation impacted the level of argumentation that occurred. These results will ultimately aid in the development and expansion of Argument-Driven Inquiry instructional models, with the goal of further enhancing students’ scientific argumentation skills and understanding of science.
IPDeN 2.0: Real-time NoC with selective flit deflection and buffering

Network-on-Chips (NoCs) have proven to be a good alternative to traditional bus-based communication architectures to interconnect all programming elements (PEs) in modern Multiprocessor Systems- on-Chips (MPSoC). Wormhole switching with Virtual-Channels (VCs) and deflection-based routing policy are the most commonly used strategies to develop NoCs for real-time systems. Deflection-based solutions have shown to be the more suitable option for systems with power and/or area constraints. However, because flits may be deflected to alternative routes when traversing the network toward their destination, their traversal times may increase. In this paper, we propose IPDeN 2.0, a new NoC that combines the benefits of buffers with the concept of deflection to avoid flits from always being deflected in contention scenarios, thus ensuring shorter communication time of flits. The proposed design also guarantees in-order flit delivery at the destination. Contrary to VCs-based NoCs, the proposed solution does not rely on expensive buffering mechanisms. Instead a single small buffer is added to each router. The buffer size is fixed and depends only on the network topology. Experimental results show that flits see their actual traversal and communication times reduced in IPDeN 2.0 as compared to IPDeN and HopliteBuf NoCs (its closest competitors) without affecting the upper-bounds on worst-case traversal and communication times. We also implemented IPDeN 2.0 using the Verilog hardware description language. We synthesize our solution for a medium-sized FPGA platform. We show that IPDeN 2.0 requires ≈ 50% less hardware resources than NoCs that rely on VCs.