Miracle-dz/UniFAR-FTUs huggingface dataset:
HADES: Hardware/Algorithm Co-design in DNN accelerators using Energy-efficient Approximate Alphabet Set Multipliers

Edge computing must be capable of executing computationally intensive algorithms, such as Deep Neural Networks (DNNs) while operating within a constrained computational resource budget. Such computations involve Matrix Vector Multiplications (MVMs) which are the dominant contributor to the memory and energy budget of DNNs. To alleviate the computational intensity and storage demand of MVMs, we propose circuit-algorithm co-design techniques with low-complexity approximate Multiply-Accumulate (MAC) units derived from the principles of Alphabet Set Multipliers (ASMs). Selection of few and proper alphabets from ASMs lead to a Multiplier-less DNN implementation, and enables encoding of low precision weights and input activations into fewer bits. To maintain accuracy under alphabet set approximations, we developed a novel ASM-alphabet aware training. The proposed low-complexity multiplication-aware algorithm was implemented In-Memory and Near-Memory with efficient shift operations to further improve the data-movement cost between memory and processing unit. We benchmark our design on CIFAR10 and ImageNet datasets for ResNet and MobileNet models and attain<1-2% accuracy degradation against full precision with energy benefits of>50% compared to standard Von-Neumann counterpart.
Recurrence With Correlation Network for Medical Image Registration

We present Recurrence with Correlation Network (RWCNet), a medical image registration network with multi-scale features and a cost volume layer. We demonstrate that these architectural features improve medical image registration accuracy in two image registration datasets prepared for the MICCAI 2022 Learn2Reg Workshop Challenge. On the large-displacement National Lung Screening Test (NLST) dataset, RWCNet is able to achieve a total registration error (TRE) of 2.11mm between corresponding keypoints without instance fine-tuning. On the OASIS brain MRI dataset, RWCNet is able to achieve an average dice overlap of 81.7% for 35 different anatomical labels. It outperforms another multi-scale network, the Laplacian Image Registration Network (LapIRN), on both datasets. Ablation experiments are performed to highlight the contribution of the various architectural features. While multi-scale features improved validation accuracy for both datasets, the cost volume layer and number of recurrent steps only improved performance on the large-displacement NLST dataset. This result suggests that cost volume layer and iterative refinement using RNN provide good support for optimization and generalization in large-displacement medical image registration. The code for RWCNet is available at https://github.com/vigsivan/optimization-based-registration.
Evolution of eccentricities induced by geometrical and quantum fluctuations in proton-nucleus collisions

We compute the azimuthal eccentricities arising from initial stage fluctuations in high energy proton-nucleus collisions at proper times $\tau \geq 0^+$. We consider two sources of fluctuations, namely the geometrical structure of the proton and the fluctuation of color fields carried by both proton and nucleus. Describing these effects with Gaussian models allows us to analytically calculate the one- and two-point correlators of energy density, from which the eccentricities are obtained. We compute the proper time evolution of these quantities by approximating the Glasma dynamics in terms of linearized Yang-Mills equations, which we solve by assuming free field propagation and adopting the dilute-dense limit.
Simulation-Driven Automated End-to-End Test and Oracle Inference

This is the first work to report on inferential testing at scale in industry. Specifically, it reports the experience of automated testing of integrity systems at Meta. We built an internal tool called ALPACAS for automated inference of end-to-end integrity tests. Integrity tests are designed to keep users safe online by checking that interventions take place when harmful behaviour occurs on a platform. ALPACAS infers not only the test input, but also the oracle, by observing production interventions to prevent harmful behaviour. This approach allows Meta to automate the process of generating integrity tests for its platforms, such as Facebook and Instagram, which consist of hundreds of millions of lines of production code. We outline the design and deployment of ALPACAS, and report results for its coverage, number of tests produced at each stage of the test inference process, and their pass rates. Specifically, we demonstrate that using ALPACAS significantly improves coverage from a manual test design for the particular aspect of integrity end-to-end testing it was applied to. Further, from a pool of 3 million data points, ALPACAS automatically yields 39 production-ready end-to-end integrity tests. We also report that the ALPACAS-inferred test suite enjoys exceptionally low flakiness for end-to-end testing with its average in-production pass rate of 99.84%.
Toward Large Kernel Models

Recent studies indicate that kernel machines can often perform similarly or better than deep neural networks (DNNs) on small datasets. The interest in kernel machines has been additionally bolstered by the discovery of their equivalence to wide neural networks in certain regimes. However, a key feature of DNNs is their ability to scale the model size and training data size independently, whereas in traditional kernel machines model size is tied to data size. Because of this coupling, scaling kernel machines to large data has been computationally challenging. In this paper, we provide a way forward for constructing large-scale general kernel models, which are a generalization of kernel machines that decouples the model and data, allowing training on large datasets. Specifically, we introduce EigenPro 3.0, an algorithm based on projected dual preconditioned SGD and show scaling to model and data sizes which have not been possible with existing kernel methods.
Universal model for electron thermal-field emission from two-dimensional semimetals

We present the theory of out-of-plane (or vertical) electron thermal-field emission from two-dimensional (2D) semimetals. We show that the current–voltage–temperature characteristic is well captured by a universal scaling relation applicable for broad classes of 2D semimetals, including graphene and its few-layer, nodal point semimetal, Dirac semimetal at the verge of topological phase transition, and nodal line semimetal. Here, an important consequence of the universal emission behavior is revealed: In contrast to the common expectation that band topology shall manifest differently in the physical observables, band topologies in two spatial dimension are indistinguishable from each other and bear no special signature in electron emission characteristics. Our findings represent the quantum extension of the universal semiclassical thermionic emission scaling law in 2D materials and provide theoretical foundations for the understanding of electron emission from cathode and charge interface transport for the design of 2D-material-based vacuum nanoelectronics.
Noise in Photoemission Current

The statistics of photoemission as computed based on a Poisson distribution of incident photons, and a binomial distribution for the photocurrent, exactly obeys a Poisson distribution so that the fluctuation in the photocurrent is just that associated with the standard deviation of a Poisson distribution. This is compared with some results in the literature.
Allocating with Priorities and Quotas: Algorithms, Complexity, and Dynamics

In many applications such as rationing medical care and supplies, university admissions, and the assignment of public housing, the decision of who receives an allocation can be justified by various normative criteria (ethical, financial, legal, etc.). Such settings have motivated the following priority-respecting allocation problem: several categories, each with a quota of interchangeable items, wish to allocate the items among a set of agents. Each category has a list of eligible agents and a priority ordering over these agents; agents may be eligible in multiple categories. The goal is to select a valid allocation: one that respects quotas, eligibility, and priorities and ensures Pareto efficiency. We provide a complete algorithmic characterization of all valid allocations, exhibiting a bijection between sets of agents who can be allocated and maximum-weight matchings under carefully chosen rank-based weights. While prior work provides a polynomial-time algorithm to locate a valid allocation, our characterization admits a simpler algorithm that enables two wide-reaching extensions: 1. Selecting valid allocations that satisfy additional criteria: Via three examples --- inclusion/exclusion of some chosen agent; agent-side Pareto efficiency vs. welfare maximization; and fairness from the perspective of allocated vs. unallocated agents --- we show that finding priority-respecting allocations subject to some secondary constraint straddles a complexity knife-edge; in each example, one problem variant can be solved efficiently, while a closely related variant is NP-hard. 2. Efficiency-envy tradeoffs in dynamic allocation: In settings where allocations must be made to T agents arriving sequentially via some stochastic process, we show that while insisting on zero priority violations leads to an Ω(T) loss in efficiency, one can design allocation policies ensuring that the sum of the efficiency loss and priority violations in hindsight is O(1) (under mild regularity conditions on the arrival process).
An Uncertainty-aware Loss Function for Training Neural Networks with Calibrated Predictions

Uncertainty quantification of machine learning and deep learning methods plays an important role in enhancing trust to the obtained result. In recent years, a numerous number of uncertainty quantification methods have been introduced. Monte Carlo dropout (MC-Dropout) is one of the most well-known techniques to quantify uncertainty in deep learning methods. In this study, we propose two new loss functions by combining cross entropy with Expected Calibration Error (ECE) and Predictive Entropy (PE). The obtained results clearly show that the new proposed loss functions lead to having a calibrated MC-Dropout method. Our results confirmed the great impact of the new hybrid loss functions for minimising the overlap between the distributions of uncertainty estimates for correct and incorrect predictions without sacrificing the model's overall performance.
Novel Fundus Image Preprocessing for Retcam Images to Improve Deep Learning Classification of Retinopathy of Prematurity

Retinopathy of Prematurity (ROP) is a potentially blinding eye disorder because of damage to the eye's retina which can affect babies born prematurely. Screening of ROP is essential for early detection and treatment. This is a laborious and manual process which requires trained physician performing dilated ophthalmological examination which can be subjective resulting in lower diagnosis success for clinically significant disease. Automated diagnostic methods can assist ophthalmologists increase diagnosis accuracy using deep learning. Several research groups have highlighted various approaches. Captured ROP Retcam images suffer from poor quality. This paper proposes the use of improved novel fundus preprocessing methods using pretrained transfer learning frameworks to create hybrid models to give higher diagnosis accuracy. Once trained and validated, the evaluations showed that these novel methods in comparison to traditional imaging processing contribute to better and in many aspects higher accuracy in classifying Plus disease, Stages of ROP and Zones in comparison to peer papers.
SpotLess: Concurrent Rotational Consensus Made Practical through Rapid View Synchronization

The emergence of blockchain technology has renewed the interest in consensus-based data management systems that are resilient to failures. To maximize the throughput of these systems, we have recently seen several prototype consensus solutions that optimize for throughput at the expense of overall implementation complexity, high costs, and reliability. Due to this, it remains unclear how these prototypes will perform in real-world environments. In this paper, we present SPOTLESS, a novel concurrent rotational consensus protocol made practical. Central to SPOTLESS is the combination of (1) a chained rotational consensus design for replicating requests with a reduced message cost and low-cost failure recovery that eliminates the traditional complex, error-prone view-change protocol; (2) the novel Rapid View Synchronization protocol that enables SPOTLESS to work in more general network assumptions, without a need for a Global Synchronization Time to synchronize view, and recover valid earlier views with the aid of non-faulty replicas without the need to rely on the primary; (3) a high-performance concurrent consensus architecture in which independent instances of the chained consensus operate concurrently to process requests with high throughput, thereby avoiding the bottlenecks seen in other rotational protocols. Due to the concurrent consensus architecture, SPOTLESS greatly outperforms traditional primary-backup consensus protocols such as PBFT (by up to 430%), NARWHAL-HS (by up to 137%), and HOTSTUFF (by up to 3803%). Due to its reduced message cost, SPOTLESS is even able to outperform RCC, a state-of-the-art high-throughput concurrent consensus protocol, by up to 23%. Furthermore, SPOTLESS is able to maintain a stable and low latency and consistently high throughput even during failures.
Cross-Fusion Rule for Personalized Federated Learning

Data scarcity and heterogeneity pose significant performance challenges for personalized federated learning, and these challenges are mainly reflected in overfitting and low precision in existing methods. To overcome these challenges, a multi-layer multi-fusion strategy framework is proposed in this paper, i.e., the server adopts the network layer parameters of each client upload model as the basic unit of fusion for information-sharing calculation. Then, a new fusion strategy combining personalized and generic is purposefully proposed, and the network layer number fusion threshold of each fusion strategy is designed according to the network layer function. Under this mechanism, the L2-Norm negative exponential similarity metric is employed to calculate the fusion weights of the corresponding feature extraction layer parameters for each client, thus improving the efficiency of heterogeneous data personalized collaboration. Meanwhile, the federated global optimal model approximation fusion strategy is adopted in the network full-connect layer, and this generic fusion strategy alleviates the overfitting introduced by forceful personalized. Finally, the experimental results show that the proposed method is superior to the state-of-the-art methods.
Deep Graph-Level Clustering Using Pseudo-Label-Guided Mutual Information Maximization Network

In this work, we study the problem of partitioning a set of graphs into different groups such that the graphs in the same group are similar while the graphs in different groups are dissimilar. This problem was rarely studied previously, although there have been a lot of work on node clustering and graph classification. The problem is challenging because it is difficult to measure the similarity or distance between graphs. One feasible approach is using graph kernels to compute a similarity matrix for the graphs and then performing spectral clustering, but the effectiveness of existing graph kernels in measuring the similarity between graphs is very limited. To solve the problem, we propose a novel method called Deep Graph-Level Clustering (DGLC). DGLC utilizes a graph isomorphism network to learn graph-level representations by maximizing the mutual information between the representations of entire graphs and substructures, under the regularization of a clustering module that ensures discriminative representations via pseudo labels. DGLC achieves graph-level representation learning and graph-level clustering in an end-to-end manner. The experimental results on six benchmark datasets of graphs show that our DGLC has state-of-the-art performance in comparison to many baselines.
PatchDCT: Patch Refinement for High Quality Instance Segmentation

High-quality instance segmentation has shown emerging importance in computer vision. Without any refinement, DCT-Mask directly generates high-resolution masks by compressed vectors. To further refine masks obtained by compressed vectors, we propose for the first time a compressed vector based multi-stage refinement framework. However, the vanilla combination does not bring significant gains, because changes in some elements of the DCT vector will affect the prediction of the entire mask. Thus, we propose a simple and novel method named PatchDCT, which separates the mask decoded from a DCT vector into several patches and refines each patch by the designed classifier and regressor. Specifically, the classifier is used to distinguish mixed patches from all patches, and to correct previously mispredicted foreground and background patches. In contrast, the regressor is used for DCT vector prediction of mixed patches, further refining the segmentation quality at boundary locations. Experiments on COCO show that our method achieves 2.0%, 3.2%, 4.5% AP and 3.4%, 5.3%, 7.0% Boundary AP improvements over Mask-RCNN on COCO, LVIS, and Cityscapes, respectively. It also surpasses DCT-Mask by 0.7%, 1.1%, 1.3% AP and 0.9%, 1.7%, 4.2% Boundary AP on COCO, LVIS and Cityscapes. Besides, the performance of PatchDCT is also competitive with other state-of-the-art methods.
TextShield: Beyond Successfully Detecting Adversarial Sentences in Text Classification

Adversarial attack serves as a major challenge for neural network models in NLP, which precludes the model's deployment in safety-critical applications. A recent line of work, detection-based defense, aims to distinguish adversarial sentences from benign ones. However, {the core limitation of previous detection methods is being incapable of giving correct predictions on adversarial sentences unlike defense methods from other paradigms. } To solve this issue, this paper proposes TextShield: (1) we discover a link between text attack and saliency information, and then we propose a saliency-based detector, which can effectively detect whether an input sentence is adversarial or not. (2) We design a saliency-based corrector, which converts the detected adversary sentences to benign ones. By combining the saliency-based detector and corrector, TextShield extends the detection-only paradigm to a detection-correction paradigm, thus filling the gap in the existing detection-based defense. Comprehensive experiments show that (a) TextShield consistently achieves higher or comparable performance than state-of-the-art defense methods across various attacks on different benchmarks. (b) our saliency-based detector outperforms existing detectors for detecting adversarial sentences.
Multivariate Time Series Anomaly Detection via Dynamic Graph Forecasting

Anomalies in univariate time series often refer to abnormal values and deviations from the temporal patterns from majority of historical observations. In multivariate time series, anomalies also refer to abnormal changes in the inter-series relationship, such as correlation, over time. Existing studies have been able to model such inter-series relationships through graph neural networks. However, most works settle on learning a static graph globally or within a context window to assist a time series forecasting task or a reconstruction task, whose objective is not tailored to explicitly detect the abnormal relationship. Some other works detect anomalies based on reconstructing or forecasting a list of inter-series graphs, which inadvertently weakens their power to capture temporal patterns within the data due to the discrete nature of graphs. In this study, we propose DyGraphAD, a multivariate time series anomaly detection framework based upon a list of dynamic inter-series graphs. The core idea is to detect anomalies based on the deviation of inter-series relationships and intra-series temporal patterns from normal to anomalous states, by leveraging the evolving nature of the graphs in order to assist a graph forecasting task and a time series forecasting task simultaneously. Our numerical experiments on real-world datasets demonstrate that DyGraphAD has superior performance than baseline anomaly detection approaches.
Updated Measurements of [O iii] 88 μm, [C ii] 158 μm, and Dust Continuum Emission from a z = 7.2 Galaxy

We present updated measurements of the [O iii] 88 μm, [C ii] 158 μm, and dust continuum emission from a star-forming galaxy at z = 7.212, SXDF-NB1006-2, by utilizing Atacama Large Millimeter/submillimeter Array (ALMA) archival data sets analysed in previous studies and data sets that have not been analysed before. The follow-up ALMA observations with higher angular resolution and sensitivity reveal a clumpy structure of the [O iii] emission on a scale of 0.32–0.85 kpc. We also combined all the ALMA [O iii] ([C ii]) data sets and updated the [O iii] ([C ii]) detection to 5. 9σ (3.6σ–4.5σ). The non-detection of [C ii] with data from the REBELS large program implies the incompleteness of spectral-scan surveys using [C ii] to detect galaxies with high star formation rates (SFRs) but marginal [C ii] emission at high-z. The dust continuum at 90 and 160 μm remains undetected, indicating little dust content of <3.9 × 106 M ⊙ (3σ), and we obtained a more stringent constraint on the total infrared luminosity. We updated the [O iii]/[C ii] luminosity ratios to 10.2 ± 4.7 (6.1 ± 3.5) and 20 ± 12 (9.6 ± 6.1) for the 4. 5σ and 3. 6σ [C ii] detections, respectively, where the ratios in the parentheses are corrected for the surface brightness dimming effect on the extended [C ii] emission. We also found a strong [C ii] deficit (0.6–1.3 dex) between SXDF-NB1006-2 and the mean L [C II]−SFR relation of galaxies at 0 < z < 9.
Consensus dynamics and coherence in hierarchical small-world networks

The hierarchical small-world network is a real-world network. It models well the benefit transmission web of the pyramid selling in China and many other countries. In this paper, by applying the spectral graph theory, we study three important aspects of the consensus problem in the hierarchical small-world network: convergence speed, communication time-delay robustness, and network coherence. Firstly, we explicitly determine the Laplacian eigenvalues of the hierarchical small-world network by making use of its treelike structure. Secondly, we find that the consensus algorithm on the hierarchical small-world network converges faster than that on some well-studied sparse networks, but is less robust to time delay. The closed-form of the first-order and the second-order network coherence are also derived. Our result shows that the hierarchical small-world network has an optimal structure of noisy consensus dynamics. Therefore, we provide a positive answer to two open questions of Yi \emph{et al}. Finally, we argue that some network structure characteristics, such as large maximum degree, small average path length, and large vertex and edge connectivity, are responsible for the strong robustness with respect to external perturbations.