Miracle-dz/UniFAR-FTUs huggingface dataset:
The all-particle cosmic ray energy spectrum measured with HAWC

Thanks to recent technological development, a new generation of cosmic ray experiments have been developed with more sensitivity to study these particles in the primary energy interval from 10 TeV to 1 PeV, such as HAWC. Due to its design and high altitude, the HAWC gamma-ray and cosmic ray observatory can provide a bridge between the data from direct and indirect cosmic ray detectors. In 2017 the HAWC collaboration published its first result on the total energy spectrum of cosmic rays, which covers the range from 10 to 500 TeV. This work updates the previous result by extending the energy interval of the measured all-particle cosmic-ray energy spectrum up to 1 PeV. The energy spectrum was obtained from the analysis of two years of HAWC's data using an unfolding method. We employed the QGSJET-II-04 model for the energy calibration and the spectrum reconstruction. The results confirm the presence of a knee like feature at tens of TeV, as previously reported by the HAWC collaboration in 2017.
RaftMLP: Do MLP-based Models Dream of Winning Over Computer Vision?

For the past ten years, CNN has reigned supreme in the world of computer vision, but recently, Transformer is on the rise. However, the quadratic computational cost of self-attention has become a severe problem of practice. There has been much research on architectures without CNN and self-attention in this context. In particular, MLP-Mixer is a simple idea designed using MLPs and hit an accuracy comparable to the Vision Transformer. However, the only inductive bias in this architecture is the embedding of tokens. Thus, there is still a possibility to build a non-convolutional inductive bias into the architecture itself, and we built in an inductive bias using two simple ideas. A way is to divide the token-mixing block vertically and horizontally. Another way is to make spatial correlations denser among some channels of token-mixing. With this approach, we were able to improve the accuracy of the MLP-Mixer while reducing its parameters and computational complexity. Compared to other MLP-based models, the proposed model, named RaftMLP has a good balance of computational complexity, the number of parameters, and actual memory usage. In addition, our work indicates that MLP-based models have the potential to replace CNNs by adopting inductive bias. The source code in PyTorch version is available at https://github.com/okojoalg/raft-mlp .
Tasks Structure Regularization in Multi-Task Learning for Improving Facial Attribute Prediction

The great success of Convolutional Neural Networks (CNN) for facial attribute prediction relies on a large amount of labeled images. Facial image datasets are usually annotated by some commonly used attributes (e.g., gender), while labels for the other attributes (e.g., big nose) are limited which causes their prediction challenging. To address this problem, we use a new Multi-Task Learning (MTL) paradigm in which a facial attribute predictor uses the knowledge of other related attributes to obtain a better generalization performance. Here, we leverage MLT paradigm in two problem settings. First, it is assumed that the structure of the tasks (e.g., grouping pattern of facial attributes) is known as a prior knowledge, and parameters of the tasks (i.e., predictors) within the same group are represented by a linear combination of a limited number of underlying basis tasks. Here, a sparsity constraint on the coefficients of this linear combination is also considered such that each task is represented in a more structured and simpler manner. Second, it is assumed that the structure of the tasks is unknown, and then structure and parameters of the tasks are learned jointly by using a Laplacian regularization framework. Our MTL methods are compared with competing methods for facial attribute prediction to show its effectiveness.
A proof of convergence for the gradient descent optimization method with random initializations in the training of neural networks with ReLU activation for piecewise linear target functions

Gradient descent (GD) type optimization methods are the standard instrument to train artificial neural networks (ANNs) with rectified linear unit (ReLU) activation. Despite the great success of GD type optimization methods in numerical simulations for the training of ANNs with ReLU activation, it remains - even in the simplest situation of the plain vanilla GD optimization method with random initializations and ANNs with one hidden layer - an open problem to prove (or disprove) the conjecture that the risk of the GD optimization method converges in the training of such ANNs to zero as the width of the ANNs, the number of independent random initializations, and the number of GD steps increase to infinity. In this article we prove this conjecture in the situation where the probability distribution of the input data is equivalent to the continuous uniform distribution on a compact interval, where the probability distributions for the random initializations of the ANN parameters are standard normal distributions, and where the target function under consideration is continuous and piecewise affine linear. Roughly speaking, the key ingredients in our mathematical convergence analysis are (i) to prove that suitable sets of global minima of the risk functions are \emph{twice continuously differentiable submanifolds of the ANN parameter spaces}, (ii) to prove that the Hessians of the risk functions on these sets of global minima satisfy an appropriate \emph{maximal rank condition}, and, thereafter, (iii) to apply the machinery in [Fehrman, B. , Gess, B. , Jentzen, A. , Convergence rates for the stochastic gradient descent method for non-convex objective functions. J. Mach. Learn. Res. 21(136): 1--48, 2020] to establish convergence of the GD optimization method with random initializations.
Γ -convergence of Onsager–Machlup functionals: I. With applications to maximum a posteriori estimation in Bayesian inverse problems

The Bayesian solution to a statistical inverse problem can be summarised by a mode of the posterior distribution, i.e. a maximum a posteriori (MAP) estimator. The MAP estimator essentially coincides with the (regularised) variational solution to the inverse problem, seen as minimisation of the Onsager–Machlup (OM) functional of the posterior measure. An open problem in the stability analysis of inverse problems is to establish a relationship between the convergence properties of solutions obtained by the variational approach and by the Bayesian approach. To address this problem, we propose a general convergence theory for modes that is based on the Γ-convergence of OM functionals, and apply this theory to Bayesian inverse problems with Gaussian and edge-preserving Besov priors. Part II of this paper considers more general prior distributions.
BEAMWAVE: Cross-layer Beamforming and Scheduling for Superimposed Transmissions in Industrial IoT mmWave Networks

The omnipresence of IoT devices in Industry 4.0 is expected to foster higher reliability, safety, and efficiency. However, interconnecting a large number of wireless devices without jeopardizing the system performance proves challenging. To address the requirements of future industries, we investigate the cross-layer design of beamforming and scheduling for layered-division multiplexing (LDM) systems in millimeter-wave bands. Scheduling is crucial as the devices in industrial settings are expected to proliferate rapidly. Also, highly performant beamforming is necessary to ensure scalability. By adopting LDM, multiple transmissions can be non-orthogonally superimposed. Specifically, we consider a superior-importance control multicast message required to be ubiquitous to all devices and inferior-importance private unicast messages targeting a subset of scheduled devices. Due to NP-hardness, we propose BEAMWAVE, which decomposes the problem into beamforming and scheduling. Through simulations, we show that BEAMWAVE attains nearoptimality and outperforms other competing schemes.
Sampling-Based Minimum Bayes Risk Decoding for Neural Machine Translation

In neural machine translation (NMT), we search for the mode of the model distribution to form predictions. The mode as well as other high probability translations found by beam search have been shown to often be inadequate in a number of ways. This prevents practitioners from improving translation quality through better search, as these idiosyncratic translations end up being selected by the decoding al-gorithm, a problem known as the beam search curse. Recently, a sampling-based approximation to minimum Bayes risk (MBR) decoding has been proposed as an alternative decision rule for NMT that would likely not suffer from the same problems. We analyse this approximation and establish that it has no equivalent to the beam search curse, i.e. better search always leads to better translations. We also design different approximations aimed at de-coupling the cost of exploration from the cost of robust estimation of expected utility. This allows for exploration of much larger hypothesis spaces, which we show to be beneﬁcial. We also show that it can be beneﬁcial to make use of strategies like beam search and nucleus sampling to construct hypothesis spaces efﬁ-ciently. We show on three language pairs (English into and from German, Romanian, and Nepali) that MBR can improve upon beam search with moderate computation. 1
End-to-End User Behavior Retrieval in Click-Through RatePrediction Model

Click-Through Rate (CTR) prediction is one of the core tasks in recommender systems (RS). It predicts a personalized click probability for each user-item pair. Recently, researchers have found that the performance of CTR model can be improved greatly by taking user behavior sequence into consideration, especially long-term user behavior sequence. The report on an e-commerce website shows that 23\% of users have more than 1000 clicks during the past 5 months. Though there are numerous works focus on modeling sequential user behaviors, few works can handle long-term user behavior sequence due to the strict inference time constraint in real world system. Two-stage methods are proposed to push the limit for better performance. At the first stage, an auxiliary task is designed to retrieve the top-$k$ similar items from long-term user behavior sequence. At the second stage, the classical attention mechanism is conducted between the candidate item and $k$ items selected in the first stage. However, information gap happens between retrieval stage and the main CTR task. This goal divergence can greatly diminishing the performance gain of long-term user sequence. In this paper, inspired by Reformer, we propose a locality-sensitive hashing (LSH) method called ETA (End-to-end Target Attention) which can greatly reduce the training and inference cost and make the end-to-end training with long-term user behavior sequence possible. Both offline and online experiments confirm the effectiveness of our model. We deploy ETA into a large-scale real world E-commerce system and achieve extra 3.1\% improvements on GMV (Gross Merchandise Value) compared to a two-stage long user sequence CTR model.
Intra-day variability of BL Lacertae from 2016 to 2018

We monitored BL Lacertae in the B, V, R and I bands for 14 nights during the period of 2016–2018. The source showed significant intra-day variability on 12 nights. We performed color-magnitude analysis and found that the source exhibited bluer-when-brighter chromatism. This bluer-when-brighter behavior is at least partly caused by the larger variation amplitude at shorter wavelengths. The variations at different wavelengths are correlated well and show no inter-band time lag.
Drell-Yan $p_{\bot}$ with NLO-matched Parton Branching TMDs at energies from fixed-target to LHC

The description of the Drell-Yan (DY) transverse momentum spectrum requires matching of fixed order QCD calculations with soft gluon resummation up to all orders in the QCD coupling. It has been noticed in the literature that a consistent description of DY data in a wide kinematic regime from fixed-target to LHC energies is problematic. In this work the predictions for transverse momentum spectrum of DY data coming from experiments in very different kinematic ranges (NuSea, R209, Phenix, LHC 8 TeV and 13 TeV center-of-mass energies \sqrt{s}s) are calculated by applying transverse momentum dependent (TMD) parton distributions obtained from the Parton Branching (PB) method, combined with the next-to-leading-order (NLO) calculation of the hard process in the MCatNLO method. We discuss the problems involved in matching of the fixed order calculation and resummation, especially in the moderate to low mass and p_{\bot}p⊥ region accessible at fixed target experiments. We find that at low DY mass and low \sqrt{s}s even in the region of p_{\bot}/Q\sim 1p⊥/Q∼1 the contribution of multiple soft gluon emissions (included in the PB-TMDs) is essential to describe the measurements, while at larger masses and LHC energies the contribution from soft gluons in the region of p_{\bot}/Q\sim 1p⊥/Q∼1 is small.
Survey and Benchmarking of Precision-Scalable MAC Arrays for Embedded DNN Processing

—Reduced-precision and variable-precision multiply-accumulate (MAC) operations provide opportunities to signiﬁ-cantly improve energy efﬁciency and throughput of DNN accelerators with no/limited algorithmic performance loss, paving a way towards deploying AI applications on resource-constraint edge devices. Accordingly, various precision-scalable MAC array (PSMA) architectures were recently proposed. However, it is difﬁcult to make a fair comparison between those alternatives, as each proposed PSMA is demonstrated in different systems with different technologies. This work aims to provide a clear view on the design space of PSMA and offer insights for selecting the optimal architectures based on designers’ needs. First, we introduce a precision-enhanced for-loop representation for DNN dataﬂows. Next, we use this new representation towards a comprehensive PSMA taxonomy, capable to systematically cover most prominent state-of-the-art PSMAs, as well as uncover new PSMA architectures. Following that, we build a highly parameterized PSMA template that can be design-time conﬁgured into a huge subset of the design space spanned by the taxonomy. This allows to fairly and thoroughly benchmark 72 different PSMA architectures. We perform such studies in 28nm technology targeting run-time precision scalability from 8 to 2 bits, operating at 200 MHz and 1 GHz. Analyzing resulting energy efﬁciency and area breakdowns reveals key design guidelines for PSMA architectures.
On the pathwise uniqueness for a class of degenerate It\^{o}-stochastic differential equations

We show pathwise uniqueness for a class of degenerate Itˆo-SDE among all of its weak solutions that spend zero time at the points of degeneracy of the dispersion matrix. Con-sequently, by the Yamada-Watanabe Theorem and a weak existence result, the pathwise unique solutions can be shown to be strong and to exist. The main tools to show pathwise uniqueness are inequalities associated with maximal functions and a Krylov type estimate derived from elliptic regularity and uniqueness in law.
Util::Lookup: Exploiting Key Decoding in Cryptographic Libraries

Implementations of cryptographic libraries have been scrutinized for secret-dependent execution behavior exploitable by microarchitectural side-channel attacks. To prevent unintended leakages, most libraries moved to constant-time implementations of cryptographic primitives. There have also been efforts to certify libraries for use in sensitive areas, like Microsoft CNG and Botan, with specific attention to leakage behavior. In this work, we show that a common oversight in these libraries is the existence of utility functions, which handle and thus possibly leak confidential information. We analyze the exploitability of base64 decoding functions across several widely used cryptographic libraries. Base64 decoding is used when loading keys stored in PEM format. We show that these functions by themselves leak sufficient information even if libraries are executed in trusted execution environments. In fact, we show that recent countermeasures to transient execution attacks such as LVI ease the exploitability of the observed faint leakages, allowing us to robustly infer sufficient information about RSA private keys with a single trace. We present a complete attack, including a broad library analysis, a high-resolution last level cache attack on SGX enclaves, and a fully parallelized implementation of the extend-and-prune approach that allows a complete key recovery at medium costs.
GANmapper: geographical content filling

We present a new method to create spatial data using a generative adversarial network (GAN). Our contribution uses coarse and widely available geospatial data to create maps of less available features at the ﬁner scale in the built environment, bypassing their traditional acquisition techniques (e.g. satellite imagery or land surveying). In the work, we employ land use data and road networks as input to generate building footprints, and conduct experiments in 9 cities around the world. The method, which we implement in a tool we release openly, enables generating approximate maps of the urban form, and it is generalisable to augment other types of geoinformation, enhancing the completeness and quality of spatial data infrastructure. It may be especially useful in locations missing detailed and high-resolution data and those that are mapped with uncertain or heterogeneous quality, such as much of OpenStreetMap. The quality of the re-sults is inﬂuenced by the urban form and scale. In most cases, experiments suggest promising performance as the method tends to truthfully indicate the locations, amount, and shape of buildings. The work has the potential to support several applications, such as energy, climate, and urban morphology studies in areas previously lacking required data.
Stochastic gravitational wave background from PBH-ABH mergers

The measurement of gravitational waves produced by binary black-hole mergers at the Advanced LIGO has encouraged extensive studies on the stochastic gravitational wave background. Recent studies have focused on gravitational wave sources made of the same species, such as mergers from binary primordial black holes or those from binary astrophysical black holes. In this paper, we study a new possibility – the stochastic gravitational wave background produced by mergers of one primordial black hole and one astrophysical black hole. Such systems are necessarily present if primordial black holes exist. We study the isotropic gravitational wave background produced through the history of the universe. We find it is very challenging to detect such a signal. We also demonstrate that it is improper to treat the gravitational waves produced by such binaries in the Milky Way as a directional stochastic background due to a very low binary formation rate.
Making Transformers Solve Compositional Tasks

Several studies have reported the inability of Transformer models to generalize compositionally, a key type of generalization in many NLP tasks such as semantic parsing. In this paper we explore the design space of Transformer models showing that the inductive biases given to the model by several design decisions significantly impact compositional generalization. We identified Transformer configurations that generalize compositionally significantly better than previously reported in the literature in many compositional tasks. We achieve state-of-the-art results in a semantic parsing compositional generalization benchmark (COGS), and a string edit operation composition benchmark (PCFG).
Quality Guidelines for Research Artifacts in Model-Driven Engineering

Sharing research artifacts is known to help people to build upon existing knowledge, adopt novel contributions in practice, and increase the chances of papers receiving attention. In Model-Driven Engineering (MDE), openly providing research artifacts plays a key role, even more so as the community targets a broader use of AI techniques, which can only become feasible if large open datasets and confidence measures for their quality are available. However, the current lack of common discipline-specific guidelines for research data sharing opens the opportunity for misunderstandings about the true potential of research artifacts and subjective expectations regarding artifact quality. To address this issue, we introduce a set of guidelines for artifact sharing specifically tailored to MDE research. To design this guidelines set, we systematically analyzed general-purpose artifact sharing practices of major computer science venues and tailored them to the MDE domain. Subsequently, we conducted an online survey with 90 researchers and practitioners with expertise in MDE. We investigated our participants’ experiences in developing and sharing artifacts in MDE research and the challenges encountered while doing so. We then asked them to prioritize each of our guidelines as essential, desirable, or unnecessary. Finally, we asked them to evaluate our guidelines with respect to clarity, completeness, and relevance. In each of these dimensions, our guidelines were assessed positively by more than 92% of the participants. To foster the reproducibility and reusability of our results, we make the full set of generated artifacts available in an open repository at https://mdeartifacts.github. io/.
FedPAGE: A Fast Local Stochastic Gradient Method for Communication-Efficient Federated Learning

Federated Averaging (FedAvg, also known as Local-SGD) (McMahan et al., 2017) is a classical federated learning algorithm in which clients run multiple local SGD steps before communicating their update to an orchestrating server. We propose a new federated learning algorithm, FedPAGE, able to further reduce the communication complexity by utilizing the recent optimal PAGE method (Li et al., 2021) instead of plain SGD in FedAvg. We show that FedPAGE uses much fewer communication rounds than previous local methods for both federated convex and nonconvex optimization. Concretely, 1) in the convex setting, the number of communication rounds of FedPAGE is $O(\frac{N^{3/4}}{S\epsilon})$, improving the best-known result $O(\frac{N}{S\epsilon})$ of SCAFFOLD (Karimireddy et al.,2020) by a factor of $N^{1/4}$, where $N$ is the total number of clients (usually is very large in federated learning), $S$ is the sampled subset of clients in each communication round, and $\epsilon$ is the target error; 2) in the nonconvex setting, the number of communication rounds of FedPAGE is $O(\frac{\sqrt{N}+S}{S\epsilon^2})$, improving the best-known result $O(\frac{N^{2/3}}{S^{2/3}\epsilon^2})$ of SCAFFOLD (Karimireddy et al.,2020) by a factor of $N^{1/6}S^{1/3}$, if the sampled clients $S\leq \sqrt{N}$. Note that in both settings, the communication cost for each round is the same for both FedPAGE and SCAFFOLD. As a result, FedPAGE achieves new state-of-the-art results in terms of communication complexity for both federated convex and nonconvex optimization.