OpenRaiser/Intern-Atlas huggingface dataset:
Conformance Constraint Discovery: Measuring Trust in Data-Driven Systems
Venue: arXiv cs.DB 2021
Year: 2021

The reliability and proper function of data-driven applications hinge on the data's continued conformance to the applications' initial design. When data deviates from this initial profile, system behavior becomes unpredictable. Data profiling techniques such as functional dependencies and denial constraints encode patterns in the data that can be used to detect deviations. But traditional methods typically focus on exact constraints and categorical attributes, and are ill-suited for tasks such as determining whether the prediction of a machine learning system can be trusted or for quantifying data drift. In this paper, we introduce data invariants, a new data-profiling primitive that models arithmetic relationships involving multiple numerical attributes within a (noisy) dataset and which complements the existing data-profiling techniques. We propose a quantitative semantics to measure the degree of violation of a data invariant, and establish that strong data invariants can be constructed from observations with low variance on the given dataset. A concrete instance of this principle gives the surprising result that low-variance components of a principal component analysis (PCA), which are usually discarded, generate better invariants than the high-variance components. We demonstrate the value of data invariants on two applications: trusted machine learning and data drift. We empirically show that data invariants can (1) reliably detect tuples on which the prediction of a machine-learned model should not be trusted, and (2) quantify data drift more accurately than the state-of-the-art methods. Additionally, we show four case studies where an intervention-centric explanation tool uses data invariants to explain causes for tuple non-conformance.
Relevance-Guided Modeling of Object Dynamics for Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Current deep reinforcement learning (RL) approaches incorporate minimal prior knowledge about the environment, limiting computational and sample efficiency. \textit{Objects} provide a succinct and causal description of the world, and many recent works have proposed unsupervised object representation learning using priors and losses over static object properties like visual consistency. However, object dynamics and interactions are also critical cues for objectness. In this paper we propose a framework for reasoning about object dynamics and behavior to rapidly determine minimal and task-specific object representations. To demonstrate the need to reason over object behavior and dynamics, we introduce a suite of RGBD MuJoCo object collection and avoidance tasks that, while intuitive and visually simple, confound state-of-the-art unsupervised object representation learning algorithms. We also highlight the potential of this framework on several Atari games, using our object representation and standard RL and planning algorithms to learn dramatically faster than existing deep RL algorithms.
Optimal Contextual Pricing and Extensions
Venue: arXiv cs.DS 2021
Year: 2021

In the contextual pricing problem a seller repeatedly obtains products described by an adversarially chosen feature vector in $\mathbb{R}^d$ and only observes the purchasing decisions of a buyer with a fixed but unknown linear valuation over the products. The regret measures the difference between the revenue the seller could have obtained knowing the buyer valuation and what can be obtained by the learning algorithm. We give a poly-time algorithm for contextual pricing with $O(d \log \log T + d \log d)$ regret which matches the $Ω(d \log \log T)$ lower bound up to the $d \log d$ additive factor. If we replace pricing loss by the symmetric loss, we obtain an algorithm with nearly optimal regret of $O(d \log d)$ matching the $Ω(d)$ lower bound up to $\log d$. These algorithms are based on a novel technique of bounding the value of the Steiner polynomial of a convex region at various scales. The Steiner polynomial is a degree $d$ polynomial with intrinsic volumes as the coefficients. We also study a generalized version of contextual search where the hidden linear function over the Euclidean space is replaced by a hidden function $f : \mathcal{X} \rightarrow \mathcal{Y}$ in a certain hypothesis class $\mathcal{H}$. We provide a generic algorithm with $O(d^2)$ regret where $d$ is the covering dimension of this class. This leads in particular to a $\tilde{O}(s^2)$ regret algorithm for linear contextual search if the linear function is guaranteed to be $s$-sparse. Finally we also extend our results to the noisy feedback model, where each round our feedback is flipped with a fixed probability $p < 1/2$.
Drone-based RGB-Infrared Cross-Modality Vehicle Detection via Uncertainty-Aware Learning
Venue: arXiv cs.CV 2021
Year: 2021

Drone-based vehicle detection aims at finding the vehicle locations and categories in an aerial image. It empowers smart city traffic management and disaster rescue. Researchers have made mount of efforts in this area and achieved considerable progress. Nevertheless, it is still a challenge when the objects are hard to distinguish, especially in low light conditions. To tackle this problem, we construct a large-scale drone-based RGB-Infrared vehicle detection dataset, termed DroneVehicle. Our DroneVehicle collects 28, 439 RGB-Infrared image pairs, covering urban roads, residential areas, parking lots, and other scenarios from day to night. Due to the great gap between RGB and infrared images, cross-modal images provide both effective information and redundant information. To address this dilemma, we further propose an uncertainty-aware cross-modality vehicle detection (UA-CMDet) framework to extract complementary information from cross-modal images, which can significantly improve the detection performance in low light conditions. An uncertainty-aware module (UAM) is designed to quantify the uncertainty weights of each modality, which is calculated by the cross-modal Intersection over Union (IoU) and the RGB illumination value. Furthermore, we design an illumination-aware cross-modal non-maximum suppression algorithm to better integrate the modal-specific information in the inference phase. Extensive experiments on the DroneVehicle dataset demonstrate the flexibility and effectiveness of the proposed method for crossmodality vehicle detection. The dataset can be download from https://github.com/VisDrone/DroneVehicle.
UAV Coverage Path Planning under Varying Power Constraints using Deep Reinforcement Learning
Venue: arXiv cs.RO 2021
Year: 2021

Coverage path planning (CPP) is the task of designing a trajectory that enables a mobile agent to travel over every point of an area of interest. We propose a new method to control an unmanned aerial vehicle (UAV) carrying a camera on a CPP mission with random start positions and multiple options for landing positions in an environment containing no-fly zones. While numerous approaches have been proposed to solve similar CPP problems, we leverage end-to-end reinforcement learning (RL) to learn a control policy that generalizes over varying power constraints for the UAV. Despite recent improvements in battery technology, the maximum flying range of small UAVs is still a severe constraint, which is exacerbated by variations in the UAV's power consumption that are hard to predict. By using map-like input channels to feed spatial information through convolutional network layers to the agent, we are able to train a double deep Q-network (DDQN) to make control decisions for the UAV, balancing limited power budget and coverage goal. The proposed method can be applied to a wide variety of environments and harmonizes complex goal structures with system constraints.
Exploiting Verified Neural Networks via Floating Point Numerical Error
Venue: arXiv cs.LG 2021
Year: 2021

Researchers have developed neural network verification algorithms motivated by the need to characterize the robustness of deep neural networks. The verifiers aspire to answer whether a neural network guarantees certain properties with respect to all inputs in a space. However, many verifiers inaccurately model floating point arithmetic but do not thoroughly discuss the consequences. We show that the negligence of floating point error leads to unsound verification that can be systematically exploited in practice. For a pretrained neural network, we present a method that efficiently searches inputs as witnesses for the incorrectness of robustness claims made by a complete verifier. We also present a method to construct neural network architectures and weights that induce wrong results of an incomplete verifier. Our results highlight that, to achieve practically reliable verification of neural networks, any verification system must accurately (or conservatively) model the effects of any floating point computations in the network inference or verification system.
DeProCams: Simultaneous Relighting, Compensation and Shape Reconstruction for Projector-Camera Systems
Venue: arXiv cs.CV 2021
Year: 2021

Image-based relighting, projector compensation and depth/normal reconstruction are three important tasks of projector-camera systems (ProCams) and spatial augmented reality (SAR). Although they share a similar pipeline of finding projector-camera image mappings, in tradition, they are addressed independently, sometimes with different prerequisites, devices and sampling images. In practice, this may be cumbersome for SAR applications to address them one-by-one. In this paper, we propose a novel end-to-end trainable model named DeProCams to explicitly learn the photometric and geometric mappings of ProCams, and once trained, DeProCams can be applied simultaneously to the three tasks. DeProCams explicitly decomposes the projector-camera image mappings into three subprocesses: shading attributes estimation, rough direct light estimation and photorealistic neural rendering. A particular challenge addressed by DeProCams is occlusion, for which we exploit epipolar constraint and propose a novel differentiable projector direct light mask. Thus, it can be learned end-to-end along with the other modules. Afterwards, to improve convergence, we apply photometric and geometric constraints such that the intermediate results are plausible. In our experiments, DeProCams shows clear advantages over previous arts with promising quality and meanwhile being fully differentiable. Moreover, by solving the three tasks in a unified model, DeProCams waives the need for additional optical devices, radiometric calibrations and structured light.
Constructing Object Groups Corresponding to Concepts for Recovery of a Summarized Sequence Diagram
Venue: arXiv cs.SE 2021
Year: 2021

Comprehending the behavior of an object-oriented system solely from its source code is troublesome, owing to its dynamism. To aid comprehension, visualizing program behavior through reverse-engineered sequence diagrams from execution traces is a promising approach. However, because of the massiveness of traces, recovered diagrams tend to become very large, causing scalability issues. To address the issues, we propose an object grouping technique that horizontally summarizes a reverse-engineered sequence diagram. Our technique constructs object groups based on Pree's meta patterns, in which each group corresponds to a concept in the domain of a subject system. Visualizing interactions only among important groups, we generate a summarized sequence diagram depicting a behavioral overview of the system. Our experiment showed that our technique outperformed the state-of-the-art trace summarization technique in terms of reducing the horizontal size of reverse-engineered sequence diagrams. Regarding the quality of object grouping, our technique achieved an F-score of 0.670 and a Recall of 0.793 on average under the condition of #lifelines (i.e., the horizontal size of a sequence diagram) < 30, whereas those of the state-of-the-art technique were 0.421 and 0.670, respectively. The runtime overhead imposed by our technique was 129.2% on average, which is relatively smaller in the literature.
Keyless Covert Communication via Channel State Information
Venue: arXiv cs.IT 2021
Year: 2021

We consider the problem of covert communication over a state-dependent channel when the channel state is available either non-causally, causally, or strictly causally, either at the transmitter alone or at both transmitter and receiver. Covert communication with respect to an adversary, called "warden," is one in which, despite communication over the channel, the warden's observation remains indistinguishable from an output induced by innocent channel-input symbols. Covert communication involves fooling an adversary in part by a proliferation of codebooks; for reliable decoding at the legitimate receiver, the codebook uncertainty is typically removed via a shared secret key that is unavailable to the warden. In contrast to previous work, we do not assume the availability of a shared key at the transmitter and legitimate receiver. Instead, shared randomness is extracted from the channel state in a manner that keeps it secret from the warden, despite the influence of the channel state on the warden's output. When channel state is available at the transmitter and receiver, we derive the covert capacity region. When channel state is only available at the transmitter, we derive inner and outer bounds on the covert capacity. We provide examples for which the covert capacity is positive with knowledge of channel state information but is zero without it.
Pixel-in-Pixel Net: Towards Efficient Facial Landmark Detection in the Wild
Venue: arXiv cs.CV 2021
Year: 2021

Recently, heatmap regression models have become popular due to their superior performance in locating facial landmarks. However, three major problems still exist among these models: (1) they are computationally expensive; (2) they usually lack explicit constraints on global shapes; (3) domain gaps are commonly present. To address these problems, we propose Pixel-in-Pixel Net (PIPNet) for facial landmark detection. The proposed model is equipped with a novel detection head based on heatmap regression, which conducts score and offset predictions simultaneously on low-resolution feature maps. By doing so, repeated upsampling layers are no longer necessary, enabling the inference time to be largely reduced without sacrificing model accuracy. Besides, a simple but effective neighbor regression module is proposed to enforce local constraints by fusing predictions from neighboring landmarks, which enhances the robustness of the new detection head. To further improve the cross-domain generalization capability of PIPNet, we propose self-training with curriculum. This training strategy is able to mine more reliable pseudo-labels from unlabeled data across domains by starting with an easier task, then gradually increasing the difficulty to provide more precise labels. Extensive experiments demonstrate the superiority of PIPNet, which obtains state-of-the-art results on three out of six popular benchmarks under the supervised setting. The results on two cross-domain test sets are also consistently improved compared to the baselines. Notably, our lightweight version of PIPNet runs at 35.7 FPS and 200 FPS on CPU and GPU, respectively, while still maintaining a competitive accuracy to state-of-the-art methods. The code of PIPNet is available at https://github.com/jhb86253817/PIPNet.
Keeping it simple: Implementation and performance of the proto-principle of adaptation and learning in the language sciences
Venue: arXiv cs.CL 2021
Year: 2021

In this paper we present the Widrow-Hoff rule and its applications to language data. After contextualizing the rule historically and placing it in the chain of neurally inspired artificial learning models, we explain its rationale and implementational considerations. Using a number of case studies we illustrate how the Widrow-Hoff rule offers unexpected opportunities for the computational simulation of a range of language phenomena that make it possible to approach old problems from a novel perspective.
Making Method of Moments Great Again? -- How can GANs learn distributions
Venue: arXiv cs.LG 2021
Year: 2021

Generative Adversarial Networks (GANs) are widely used models to learn complex real-world distributions. In GANs, the training of the generator usually stops when the discriminator can no longer distinguish the generator's output from the set of training examples. A central question of GANs is that when the training stops, whether the generated distribution is actually close to the target distribution, and how the training process reaches to such configurations efficiently? In this paper, we established a theoretical results towards understanding this generator-discriminator training process. We empirically observe that during the earlier stage of the GANs training, the discriminator is trying to force the generator to match the low degree moments between the generator's output and the target distribution. Moreover, only by matching these empirical moments over polynomially many training examples, we prove that the generator can already learn notable class of distributions, including those that can be generated by two-layer neural networks.
Adjacency Labelling for Planar Graphs (and Beyond)
Venue: arXiv cs.DS 2021
Year: 2021

We show that there exists an adjacency labelling scheme for planar graphs where each vertex of an $n$-vertex planar graph $G$ is assigned a $(1+o(1))\log_2 n$-bit label and the labels of two vertices $u$ and $v$ are sufficient to determine if $uv$ is an edge of $G$. This is optimal up to the lower order term and is the first such asymptotically optimal result. An alternative, but equivalent, interpretation of this result is that, for every $n$, there exists a graph $U_n$ with $n^{1+o(1)}$ vertices such that every $n$-vertex planar graph is an induced subgraph of $U_n$. These results generalize to bounded genus graphs, apex-minor-free graphs, bounded-degree graphs from minor closed families, and $k$-planar graphs.
Designing constraint-based false data injection attacks against the unbalanced distribution smart grids
Venue: arXiv cs.CR 2021
Year: 2021

The advent of smart power grid which plays a vital role in the upcoming smart city era is accompanied with the implementation of a monitoring tool, called state estimation. For the case of the unbalanced residential distribution grid, the state estimating operation which is conducted at a regional scale is considered as an application of the edge computing-based Internet of Things (IoT). While the outcome of the state estimation is important to the subsequent control activities, its accuracy heavily depends on the data integrity of the information collected from the scattered measurement devices. This fact exposes the vulnerability of the state estimation module under the effect of data-driven attacks. Among these, false data injection attack (FDI) is attracting much attention due to its capability to interfere with the normal operation of the network without being detected. This paper presents an attack design scheme based on a nonlinear physical-constraint model that is able to produce an FDI attack with theoretically stealthy characteristic. To demonstrate the effectiveness of the proposed design scheme, simulations with the IEEE 13-node test feeder and the WSCC 9-bus system are conducted. The experimental results indicate that not only the false positive rate of the bad data detection mechanism is 100 per cent but the physical consequence of the attack is severe. These results pose a serious challenge for operators in maintaining the integrity of measurement data.
Improving the Backpropagation Algorithm with Consequentialism Weight Updates over Mini-Batches
Venue: arXiv cs.LG 2021
Year: 2021

Many attempts took place to improve the adaptive filters that can also be useful to improve backpropagation (BP). Normalized least mean squares (NLMS) is one of the most successful algorithms derived from Least mean squares (LMS). However, its extension to multi-layer neural networks has not happened before. Here, we first show that it is possible to consider a multi-layer neural network as a stack of adaptive filters. Additionally, we introduce more comprehensible interpretations of NLMS than the complicated geometric interpretation in affine projection algorithm (APA) for a single fully-connected (FC) layer that can easily be generalized to, for instance, convolutional neural networks and also works better with mini-batch training. With this new viewpoint, we introduce a better algorithm by predicting then emending the adverse consequences of the actions that take place in BP even before they happen. Finally, the proposed method is compatible with stochastic gradient descent (SGD) and applicable to momentum-based derivatives such as RMSProp, Adam, and NAG. Our experiments show the usefulness of our algorithm in the training of deep neural networks.
Developing and Operating Artificial Intelligence Models in Trustworthy Autonomous Systems
Venue: arXiv cs.SE 2021
Year: 2021

Companies dealing with Artificial Intelligence (AI) models in Autonomous Systems (AS) face several problems, such as users' lack of trust in adverse or unknown conditions, gaps between software engineering and AI model development, and operation in a continuously changing operational environment. This work-in-progress paper aims to close the gap between the development and operation of trustworthy AI-based AS by defining an approach that coordinates both activities. We synthesize the main challenges of AI-based AS in industrial settings. We reflect on the research efforts required to overcome these challenges and propose a novel, holistic DevOps approach to put it into practice. We elaborate on four research directions: (a) increased users' trust by monitoring operational AI-based AS and identifying self-adaptation needs in critical situations; (b) integrated agile process for the development and evolution of AI models and AS; (c) continuous deployment of different context-specific instances of AI models in a distributed setting of AS; and (d) holistic DevOps-based lifecycle for AI-based AS.
Bypassing memory safety mechanisms through speculative control flow hijacks
Venue: arXiv cs.CR 2021
Year: 2021

The prevalence of memory corruption bugs in the past decades resulted in numerous defenses, such as stack canaries, control flow integrity (CFI), and memory safe languages. These defenses can prevent entire classes of vulnerabilities, and help increase the security posture of a program. In this paper, we show that memory corruption defenses can be bypassed using speculative execution attacks. We study the cases of stack protectors, CFI, and bounds checks in Go, demonstrating under which conditions they can be bypassed by a form of speculative control flow hijack, relying on speculative or architectural overwrites of control flow data. Information is leaked by redirecting the speculative control flow of the victim to a gadget accessing secret data and acting as a side channel send. We also demonstrate, for the first time, that this can be achieved by stitching together multiple gadgets, in a speculative return-oriented programming attack. We discuss and implement software mitigations, showing moderate performance impact.