OpenRaiser/Intern-Atlas huggingface dataset:
Power Normalizations in Fine-grained Image, Few-shot Image and Graph Classification
Venue: arXiv cs.CV 2021
Year: 2021

Power Normalizations (PN) are useful non-linear operators which tackle feature imbalances in classification problems. We study PNs in the deep learning setup via a novel PN layer pooling feature maps. Our layer combines the feature vectors and their respective spatial locations in the feature maps produced by the last convolutional layer of CNN into a positive definite matrix with second-order statistics to which PN operators are applied, forming so-called Second-order Pooling (SOP). As the main goal of this paper is to study Power Normalizations, we investigate the role and meaning of MaxExp and Gamma, two popular PN functions. To this end, we provide probabilistic interpretations of such element-wise operators and discover surrogates with well-behaved derivatives for end-to-end training. Furthermore, we look at the spectral applicability of MaxExp and Gamma by studying Spectral Power Normalizations (SPN). We show that SPN on the autocorrelation/covariance matrix and the Heat Diffusion Process (HDP) on a graph Laplacian matrix are closely related, thus sharing their properties. Such a finding leads us to the culmination of our work, a fast spectral MaxExp which is a variant of HDP for covariances/autocorrelation matrices. We evaluate our ideas on fine-grained recognition, scene recognition, and material classification, as well as in few-shot learning and graph classification.
Efficient Randomized Distributed Coloring in CONGEST
Venue: arXiv cs.DS 2021
Year: 2021

Distributed vertex coloring is one of the classic problems and probably also the most widely studied problems in the area of distributed graph algorithms. We present a new randomized distributed vertex coloring algorithm for the standard CONGEST model, where the network is modeled as an $n$-node graph $G$, and where the nodes of $G$ operate in synchronous communication rounds in which they can exchange $O(\log n)$-bit messages over all the edges of $G$. For graphs with maximum degree $Δ$, we show that the $(Δ+1)$-list coloring problem (and therefore also the standard $(Δ+1)$-coloring problem) can be solved in $O(\log^5\log n)$ rounds. Previously such a result was only known for the significantly more powerful LOCAL model, where in each round, neighboring nodes can exchange messages of arbitrary size. The best previous $(Δ+1)$-coloring algorithm in the CONGEST model had a running time of $O(\logΔ+ \log^6\log n)$ rounds. As a function of $n$ alone, the best previous algorithm therefore had a round complexity of $O(\log n)$, which is a bound that can also be achieved by a naïve folklore algorithm. For large maximum degree $Δ$, our algorithm hence is an exponential improvement over the previous state of the art.
IRO: Integrity and Reliability Enhanced Ring ORAM
Venue: arXiv cs.CR 2021
Year: 2021

Memory security and reliability are two of the major design concerns in cloud computing systems. State-of-the-art memory security-reliability co-designs (e.g. Synergy) have achieved a good balance on performance, confidentiality, integrity, and reliability. However, these works merely rely on encryption to ensure data confidentiality, which has been proven unable to prevent information leakage from memory access patterns. Ring ORAM is an attractive confidential protection protocol to hide memory access patterns to the untrusted storage system. Unfortunately, it does not compatible with the security-reliability co-designs. A forced combination would result in more severe performance loss. In this paper, we propose IRO, an Integrity and Reliability enhanced Ring ORAM design. To reduce the overhead of integrity verification, we propose a low overhead integrity tree RIT and use a Minimum Update Subtree Tree (MUST) to reduce metadata update overhead. To improve memory reliability, we present Secure Replication to provide channel-level error resilience for the ORAM tree and use the mirrored channel technique to guarantee the reliability of the MUST. Last, we use the error correction pointer (ECP) to repair permanent memory cell fault to further improve device reliability and lifetime. A compact metadata design is used to reduce the storage and consulting overhead of the ECP. IRO provides strong security and reliability guarantees, while the resulting storage and performance overhead is very small. Our evaluation shows that IRO only increases 7.54% execution time on average over the Baseline under two channels four AES-GCM units setting. With enough AES-GCM units to perform concurrent MAC computing, IRO can reduce 2.14% execution time of the Baseline.
Unifying Decidable Entailments in Separation Logic with Inductive Definitions
Venue: arXiv cs.LO 2021
Year: 2021

The entailment problem $φ\models ψ$ in Separation Logic \cite{IshtiaqOHearn01,Reynolds02}, between separated conjunctions of equational ($x \iseq y$ and $x \not\iseq y$), spatial ($x \mapsto (y_1,\ldots,y_\rank)$) and predicate ($p(x_1,\ldots,x_n)$) atoms, interpreted by a finite set of inductive rules, is undecidable in general. Certain restrictions on the set of inductive definitions lead to decidable classes of entailment problems. Currently, there are two such decidable classes, based on two restrictions, called \emph{establishment} \cite{IosifRogalewiczSimacek13,KatelaanMathejaZuleger19,PZ20} and \emph{restrictedness} \cite{EIP21a}, respectively. Both classes are shown to be in \twoexptime\ by the independent proofs from \cite{PZ20} and \cite{EIP21a}, respectively, and a many-one reduction of established to restricted entailment problems has been given \cite{EIP21a}. In this paper, we strictly generalize the restricted class, by distinguishing the conditions that apply only to the left- ($φ$) and the right- ($ψ$) hand side of entailments, respectively. We provide a many-one reduction of this generalized class, called \emph{safe}, to the established class. Together with the reduction of established to restricted entailment problems, this new reduction closes the loop and shows that the three classes of entailment problems (respectively established, restricted and safe) form a single, unified, \twoexptime-complete class.
A circular version of Gödel's T and its abstraction complexity
Venue: arXiv cs.LO 2021
Year: 2021

Circular and non-wellfounded proofs have become an increasingly popular tool for metalogical treatments of systems with forms of induction and/or recursion. In this work we investigate the expressivity of a variant CT of Gödel's system T where programs are circularly typed, rather than including an explicit recursion combinator. In particular, we examine the abstraction complexity (i.e. type level) of C, and show that the Gödel primitive recursive functionals may be typed more succinctly with circular derivations, using types precisely one level lower than in T. In fact we give a logical correspondence between the two settings, interpreting the quantifier-free type 1 theory of level n+1 T into that of level n C and vice-versa. We also obtain some further results and perspectives on circular 'derivations', namely strong normalisation and confluence, models based on hereditary computable functionals, continuity at type 2, and a translation to terms of $\T$ computing the same functional, at all types.
A Paragraph-level Multi-task Learning Model for Scientific Fact-Verification
Venue: arXiv cs.CL 2021
Year: 2021

Even for domain experts, it is a non-trivial task to verify a scientific claim by providing supporting or refuting evidence rationales. The situation worsens as misinformation is proliferated on social media or news websites, manually or programmatically, at every moment. As a result, an automatic fact-verification tool becomes crucial for combating the spread of misinformation. In this work, we propose a novel, paragraph-level, multi-task learning model for the SciFact task by directly computing a sequence of contextualized sentence embeddings from a BERT model and jointly training the model on rationale selection and stance prediction.
NBR: Neutralization Based Reclamation
Venue: arXiv cs.DC 2021
Year: 2021

Safe memory reclamation (SMR) algorithms suffer from a trade-off between bounding unreclaimed memory and the speed of reclamation. Hazard pointer (HP) based algorithms bound unreclaimed memory at all times, but tend to be slower than other approaches. Epoch based reclamation (EBR) algorithms are faster, but do not bound memory reclamation. Other algorithms follow hybrid approaches, requiring special compiler or hardware support, changes to record layouts, and/or extensive code changes. Not all SMR algorithms can be used to reclaim memory for all data structures. We propose a new neutralization based reclamation (NBR) algorithm that is faster than the best known EBR algorithms and achieves bounded unreclaimed memory. It is non-blocking when used with a non-blocking operating system (OS) kernel, and only requires atomic read, write and CAS. NBR is straightforward to use with many different data structures, and in most cases, require similar reasoning and programmer effort to two-phased locking. NBR is implemented using OS signals and a lightweight handshaking mechanism between participating threads to determine when it is safe to reclaim a record. Experiments on a lock-based binary search tree and a lazy linked list show that NBR significantly outperforms many state of the art reclamation algorithms. In the tree NBR is faster than next best algorithm, DEBRA by upto 38% and HP by upto 17%. And, in the list NBR is 15% and 243% faster than DEBRA and HP, respectively.
Selling Renewable Utilization Service to Consumers via Cloud Energy Storage
Venue: arXiv cs.GT 2021
Year: 2021

This paper proposes a cloud energy storage (CES) model for enabling local renewable integration of building consumers (BCs). Different from most existing third-party based ES sharing models that the energy storage operator (ESO) gains profit by leasing energy or power capacity, our CES model allows the ESO to sell renewable utilization service (RUS) to its consumers, i.e., the total amount of local renewable generation shifted to supply their demand. Notably, we propose a quadratic price model for the ESO charging its consumers by their requested RUS and formulate their interactions as a Stackelberg game, which admits an equilibrium. We prove the CES model outperforms individual ES (IES) model in social welfare. Besides, we study the performance of the CES model compared with the IES model and an existing ES sharing model (referring to VES model) via case studies. We demonstrate the CES model can provide 2-4 times profit to the ESO than the VES model. Meanwhile, higher cost reduction for the BCs are secured by the CES model. Moreover, we show the CES model can achieve near social optima and high ES efficiency (i.e., utilization) which are not provided by the other ES models.
With False Friends Like These, Who Can Notice Mistakes?
Venue: arXiv cs.LG 2021
Year: 2021

Adversarial examples crafted by an explicit adversary have attracted significant attention in machine learning. However, the security risk posed by a potential false friend has been largely overlooked. In this paper, we unveil the threat of hypocritical examples -- inputs that are originally misclassified yet perturbed by a false friend to force correct predictions. While such perturbed examples seem harmless, we point out for the first time that they could be maliciously used to conceal the mistakes of a substandard (i.e., not as good as required) model during an evaluation. Once a deployer trusts the hypocritical performance and applies the "well-performed" model in real-world applications, unexpected failures may happen even in benign environments. More seriously, this security risk seems to be pervasive: we find that many types of substandard models are vulnerable to hypocritical examples across multiple datasets. Furthermore, we provide the first attempt to characterize the threat with a metric called hypocritical risk and try to circumvent it via several countermeasures. Results demonstrate the effectiveness of the countermeasures, while the risk remains non-negligible even after adaptive robust training.
Dynamic Graph-Based Anomaly Detection in the Electrical Grid
Venue: arXiv cs.LG 2021
Year: 2021

Given sensor readings over time from a power grid, how can we accurately detect when an anomaly occurs? A key part of achieving this goal is to use the network of power grid sensors to quickly detect, in real-time, when any unusual events, whether natural faults or malicious, occur on the power grid. Existing bad-data detectors in the industry lack the sophistication to robustly detect broad types of anomalies, especially those due to emerging cyber-attacks, since they operate on a single measurement snapshot of the grid at a time. New ML methods are more widely applicable, but generally do not consider the impact of topology change on sensor measurements and thus cannot accommodate regular topology adjustments in historical data. Hence, we propose DYNWATCH, a domain knowledge based and topology-aware algorithm for anomaly detection using sensors placed on a dynamic grid. Our approach is accurate, outperforming existing approaches by 20% or more (F-measure) in experiments; and fast, running in less than 1.7ms on average per time tick per sensor on a 60K+ branch case using a laptop computer, and scaling linearly in the size of the graph.
Ensembles of Localised Models for Time Series Forecasting
Venue: arXiv cs.LG 2021
Year: 2021

With large quantities of data typically available nowadays, forecasting models that are trained across sets of time series, known as Global Forecasting Models (GFM), are regularly outperforming traditional univariate forecasting models that work on isolated series. As GFMs usually share the same set of parameters across all time series, they often have the problem of not being localised enough to a particular series, especially in situations where datasets are heterogeneous. We study how ensembling techniques can be used with generic GFMs and univariate models to solve this issue. Our work systematises and compares relevant current approaches, namely clustering series and training separate submodels per cluster, the so-called ensemble of specialists approach, and building heterogeneous ensembles of global and local models. We fill some gaps in the existing GFM localisation approaches, in particular by incorporating varied clustering techniques such as feature-based clustering, distance-based clustering and random clustering, and generalise them to use different underlying GFM model types. We then propose a new methodology of clustered ensembles where we train multiple GFMs on different clusters of series, obtained by changing the number of clusters and cluster seeds. Using Feed-forward Neural Networks, Recurrent Neural Networks, and Pooled Regression models as the underlying GFMs, in our evaluation on eight publicly available datasets, the proposed models are able to achieve significantly higher accuracy than baseline GFM models and univariate forecasting methods.
Patch-wise++ Perturbation for Adversarial Targeted Attacks
Venue: arXiv cs.CV 2021
Year: 2021

Although great progress has been made on adversarial attacks for deep neural networks (DNNs), their transferability is still unsatisfactory, especially for targeted attacks. There are two problems behind that have been long overlooked: 1) the conventional setting of $T$ iterations with the step size of $ε/T$ to comply with the $ε$-constraint. In this case, most of the pixels are allowed to add very small noise, much less than $ε$; and 2) usually manipulating pixel-wise noise. However, features of a pixel extracted by DNNs are influenced by its surrounding regions, and different DNNs generally focus on different discriminative regions in recognition. To tackle these issues, our previous work proposes a patch-wise iterative method (PIM) aimed at crafting adversarial examples with high transferability. Specifically, we introduce an amplification factor to the step size in each iteration, and one pixel's overall gradient overflowing the $ε$-constraint is properly assigned to its surrounding regions by a project kernel. But targeted attacks aim to push the adversarial examples into the territory of a specific class, and the amplification factor may lead to underfitting. Thus, we introduce the temperature and propose a patch-wise++ iterative method (PIM++) to further improve transferability without significantly sacrificing the performance of the white-box attack. Our method can be generally integrated to any gradient-based attack methods. Compared with the current state-of-the-art attack methods, we significantly improve the success rate by 33.1\% for defense models and 31.4\% for normally trained models on average.
AraGPT2: Pre-Trained Transformer for Arabic Language Generation
Venue: arXiv cs.CL 2021
Year: 2021

Recently, pre-trained transformer-based architectures have proven to be very efficient at language modeling and understanding, given that they are trained on a large enough corpus. Applications in language generation for Arabic are still lagging in comparison to other NLP advances primarily due to the lack of advanced Arabic language generation models. In this paper, we develop the first advanced Arabic language generation model, AraGPT2, trained from scratch on a large Arabic corpus of internet text and news articles. Our largest model, AraGPT2-mega, has 1.46 billion parameters, which makes it the largest Arabic language model available. The Mega model was evaluated and showed success on different tasks including synthetic news generation, and zero-shot question answering. For text generation, our best model achieves a perplexity of 29.8 on held-out Wikipedia articles. A study conducted with human evaluators showed the significant success of AraGPT2-mega in generating news articles that are difficult to distinguish from articles written by humans. We thus develop and release an automatic discriminator model with a 98% percent accuracy in detecting model-generated text. The models are also publicly available, hoping to encourage new research directions and applications for Arabic NLP.
xERTE: Explainable Reasoning on Temporal Knowledge Graphs for Forecasting Future Links
Venue: arXiv cs.LG 2021
Year: 2021

Modeling time-evolving knowledge graphs (KGs) has recently gained increasing interest. Here, graph representation learning has become the dominant paradigm for link prediction on temporal KGs. However, the embedding-based approaches largely operate in a black-box fashion, lacking the ability to interpret their predictions. This paper provides a link forecasting framework that reasons over query-relevant subgraphs of temporal KGs and jointly models the structural dependencies and the temporal dynamics. Especially, we propose a temporal relational attention mechanism and a novel reverse representation update scheme to guide the extraction of an enclosing subgraph around the query. The subgraph is expanded by an iterative sampling of temporal neighbors and by attention propagation. Our approach provides human-understandable evidence explaining the forecast. We evaluate our model on four benchmark temporal knowledge graphs for the link forecasting task. While being more explainable, our model obtains a relative improvement of up to 20% on Hits@1 compared to the previous best KG forecasting method. We also conduct a survey with 53 respondents, and the results show that the evidence extracted by the model for link forecasting is aligned with human understanding.
Unsupervised Monocular Depth Reconstruction of Non-Rigid Scenes
Venue: arXiv cs.CV 2021
Year: 2021

Monocular depth reconstruction of complex and dynamic scenes is a highly challenging problem. While for rigid scenes learning-based methods have been offering promising results even in unsupervised cases, there exists little to no literature addressing the same for dynamic and deformable scenes. In this work, we present an unsupervised monocular framework for dense depth estimation of dynamic scenes, which jointly reconstructs rigid and non-rigid parts without explicitly modelling the camera motion. Using dense correspondences, we derive a training objective that aims to opportunistically preserve pairwise distances between reconstructed 3D points. In this process, the dense depth map is learned implicitly using the as-rigid-as-possible hypothesis. Our method provides promising results, demonstrating its capability of reconstructing 3D from challenging videos of non-rigid scenes. Furthermore, the proposed method also provides unsupervised motion segmentation results as an auxiliary output.
A Detailed Account of The Inconsistent Labelling Problem of Stutter-Preserving Partial-Order Reduction
Venue: arXiv cs.LO 2021
Year: 2021

One of the most popular state-space reduction techniques for model checking is partial-order reduction (POR). Of the many different POR implementations, stubborn sets are a very versatile variant and have thus seen many different applications over the past 32 years. One of the early stubborn sets works shows how the basic conditions for reduction can be augmented to preserve stutter-trace equivalence, making stubborn sets suitable for model checking of linear-time properties. In this paper, we identify a flaw in the reasoning and show with a counter-example that stutter-trace equivalence is not necessarily preserved. We propose a stronger reduction condition and provide extensive new correctness proofs to ensure the issue is resolved. Furthermore, we analyse in which formalisms the problem may occur. The impact on practical implementations is limited, since they all compute a correct approximation of the theory.
Bayesian Graph Neural Network for Fast identification of critical nodes in Uncertain Complex Networks
Venue: arXiv cs.SI 2021
Year: 2021

In the quest to improve efficiency, interdependence and complexity are becoming defining characteristics of modern complex networks representing engineered and natural systems. Graph theory is a widely used framework for modeling such complex networks and to evaluate their robustness to disruptions. Particularly, identification of critical nodes/links in a graph can facilitate the enhancement of graph (system) robustness and characterize crucial factors of system performance. Most existing methods of critical node identification are based on an iterative approach that explores each node/link of a graph. These methods suffer from high computational complexity and the resulting analysis is network specific. Additionally, uncertainty associated with the underlying graphical model further limits the potential value of these traditional approaches. To overcome these challenges, we propose a Bayesian graph neural network based node classification framework that is computationally efficient and systematically incorporates uncertainties. Instead of utilizing the observed graph for training the model, a MAP estimate of the graph is computed based on the observed topology and node target labels. Further, a Monte-Carlo (MC) dropout algorithm is incorporated to account for the epistemic uncertainty. The fidelity and the gain in computational complexity offered by the Bayesian framework is illustrated using simulation results.