OpenRaiser/Intern-Atlas huggingface dataset:
Classifier-Free Diffusion Guidance
Venue: arXiv cs.LG 2022
Year: 2022

Classifier guidance is a recently introduced method to trade off mode coverage and sample fidelity in conditional diffusion models post training, in the same spirit as low temperature sampling or truncation in other types of generative models. Classifier guidance combines the score estimate of a diffusion model with the gradient of an image classifier and thereby requires training an image classifier separate from the diffusion model. It also raises the question of whether guidance can be performed without a classifier. We show that guidance can be indeed performed by a pure generative model without such a classifier: in what we call classifier-free guidance, we jointly train a conditional and an unconditional diffusion model, and we combine the resulting conditional and unconditional score estimates to attain a trade-off between sample quality and diversity similar to that obtained using classifier guidance.
Scalable Cyber-Physical Testbed for Cybersecurity Evaluation of Synchrophasors in Power Systems
Venue: arXiv cs.CR 2022
Year: 2022

This paper presents a real-time cyber-physical (CPS) testbed for power systems with different real attack scenarios on the synchrophasors-phasor measurement units (PMU). The testbed focuses on real-time cyber-security emulation with components including a digital real-time simulator, virtual machines (VM), a communication network emulator, and a package manipulation tool. The script-based VM deployment and the software-defined network emulation facilitate a highly-scalable cyber-physical testbed, which enables emulations of a real power system under different attack scenarios such as Address Resolution Protocol (ARP) poisoning attack, Man In The Middle (MITM) attack, False Data Injection Attack (FDIA), and Eavesdropping Attack. The common synchrophasor, IEEE C37.118.2 named pySynphasor has been implemented and analyzed for its security vulnerabilities. The paper also presented an interactive framework for injecting false data into a realistic system utilizing the pySynphasor module. The framework can dissect and reconstruct the C37.118.2 packets, which expands the potential of testing and developing PMU-based systems and their security in detail and benefits the power industry and academia. A case for the demonstration of the FDIA attack on the linear state estimation together with the bad-data detection procedure are presented as an example of the testbed capability.
Multi-Attention Network for Compressed Video Referring Object Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Referring video object segmentation aims to segment the object referred by a given language expression. Existing works typically require compressed video bitstream to be decoded to RGB frames before being segmented, which increases computation and storage requirements and ultimately slows the inference down. This may hamper its application in real-world computing resource limited scenarios, such as autonomous cars and drones. To alleviate this problem, in this paper, we explore the referring object segmentation task on compressed videos, namely on the original video data flow. Besides the inherent difficulty of the video referring object segmentation task itself, obtaining discriminative representation from compressed video is also rather challenging. To address this problem, we propose a multi-attention network which consists of dual-path dual-attention module and a query-based cross-modal Transformer module. Specifically, the dual-path dual-attention module is designed to extract effective representation from compressed data in three modalities, i.e., I-frame, Motion Vector and Residual. The query-based cross-modal Transformer firstly models the correlation between linguistic and visual modalities, and then the fused multi-modality features are used to guide object queries to generate a content-aware dynamic kernel and to predict final segmentation masks. Different from previous works, we propose to learn just one kernel, which thus removes the complicated post mask-matching procedure of existing methods. Extensive promising experimental results on three challenging datasets show the effectiveness of our method compared against several state-of-the-art methods which are proposed for processing RGB data. Source code is available at: https://github.com/DexiangHong/MANet.
Learning Bipedal Walking On Planned Footsteps For Humanoid Robots
Venue: arXiv cs.RO 2022
Year: 2022

Deep reinforcement learning (RL) based controllers for legged robots have demonstrated impressive robustness for walking in different environments for several robot platforms. To enable the application of RL policies for humanoid robots in real-world settings, it is crucial to build a system that can achieve robust walking in any direction, on 2D and 3D terrains, and be controllable by a user-command. In this paper, we tackle this problem by learning a policy to follow a given step sequence. The policy is trained with the help of a set of procedurally generated step sequences (also called footstep plans). We show that simply feeding the upcoming 2 steps to the policy is sufficient to achieve omnidirectional walking, turning in place, standing, and climbing stairs. Our method employs curriculum learning on the complexity of terrains, and circumvents the need for reference motions or pre-trained weights. We demonstrate the application of our proposed method to learn RL policies for 2 new robot platforms - HRP5P and JVRC-1 - in the MuJoCo simulation environment. The code for training and evaluation is available online.
Design of Classes I
Venue: arXiv cs.CY 2022
Year: 2022

The use of functional programming languages in the first programming course at many universities is well-established and effective. Invariably, however, students must progress to study object-oriented programming. This article presents how the first steps of this transition have been successfully implemented at Seton Hall University. The developed methodology builds on the students' experience with type-based design acquired in their previous introduction to programming courses. The transition is made smooth by explicitly showing students that the design lessons they have internalized are relevant in object-oriented programming. This allows for new abstractions offered by object-oriented programming languages to be more easily taught and used by students. Empirical evidence collected from students in the course suggests that the approach developed is effective and that the transition is smooth.
Reimplementing the Wheel: Teaching Compilers with a Small Self-Contained One
Venue: arXiv cs.PL 2022
Year: 2022

We report on a one-semester compiler construction course based on the idea of implementing a small self-contained compiler for a small model language from scratch, not using other compiler construction frameworks. The course is built around an evolving family of languages with increasing expressiveness and complexity, which finally is crowned by a language with first-class functions, S-expressions, pattern matching, and garbage collection. The code generation technique is based on the idea of symbolic interpreters, which allows to implement a robust albeit not a very efficient native code generator. We give the motivation for the course, describe its structure, and report some results of teaching based on students' post-course surveys.
On the Interaction between Test-Suite Reduction and Regression-Test Selection Strategies
Venue: arXiv cs.SE 2022
Year: 2022

Unit testing is one of the most established quality-assurance techniques for software development. One major advantage of unit testing is the adjustable trade-off between efficiency (i.e., testing effort) and effectiveness (i.e., fault-detection probability). To this end, various strategies have been proposed to exploit this trade-off. In particular, test-suite reduction (TSR) reduces the number of (presumably redundant) test cases while testing a single program version. Regression-test selection (RTS) selects test cases for testing consecutive program revisions. However, both TSR and RTS may influence -- or even obstruct -- each others' performance when used in combination. For instance, test cases discarded during TSR for a particular program version may become relevant again for RTS. However, finding a combination of both strategies leading to a reasonable trade-off throughout the version history of a program is an open question. The goal of this paper is to gain a better understanding of the interactions between TSR and RTS with respect to efficiency and effectiveness. To this end, we present a configurable framework called RegreTS for automated unit-testing of C programs. The framework comprises different strategies for TSR and RTS and possible combinations thereof. We apply this framework to a collection of subject systems, delivering several crucial insights. First, TSR has almost always a negative impact on the effectiveness of RTS, yet a positive impact on efficiency. Second, test cases revealing to testers the effect of program modifications between consecutive program versions are far more effective than test cases simply covering modified code parts, yet causing much more testing effort.
An exhaustive variable selection study for linear models of soundscape emotions: rankings and Gibbs analysis
Venue: arXiv cs.SD 2022
Year: 2022

In the last decade, soundscapes have become one of the most active topics in Acoustics, providing a holistic approach to the acoustic environment, which involves human perception and context. Soundscapes-elicited emotions are central and substantially subtle and unnoticed (compared to speech or music). Currently, soundscape emotion recognition is a very active topic in the literature. We provide an exhaustive variable selection study (i.e., a selection of the soundscapes indicators) to a well-known dataset (emo-soundscapes). We consider linear soundscape emotion models for two soundscapes descriptors: arousal and valence. Several ranking schemes and procedures for selecting the number of variables are applied. We have also performed an alternating optimization scheme for obtaining the best sequences keeping fixed a certain number of features. Furthermore, we have designed a novel technique based on Gibbs sampling, which provides a more complete and clear view of the relevance of each variable. Finally, we have also compared our results with the analysis obtained by the classical methods based on p-values. As a result of our study, we suggest two simple and parsimonious linear models of only 7 and 16 variables (within the 122 possible features) for the two outputs (arousal and valence), respectively. The suggested linear models provide very good and competitive performance, with $R^2>0.86$ and $R^2>0.63$ (values obtained after a cross-validation procedure), respectively.
Quiver neural networks
Venue: arXiv cs.LG 2022
Year: 2022

We develop a uniform theoretical approach towards the analysis of various neural network connectivity architectures by introducing the notion of a quiver neural network. Inspired by quiver representation theory in mathematics, this approach gives a compact way to capture elaborate data flows in complex network architectures. As an application, we use parameter space symmetries to prove a lossless model compression algorithm for quiver neural networks with certain non-pointwise activations known as rescaling activations. In the case of radial rescaling activations, we prove that training the compressed model with gradient descent is equivalent to training the original model with projected gradient descent.
Multi-cell Content Caching: Optimization for Cost and Information Freshness
Venue: arXiv cs.IT 2022
Year: 2022

In multi-access edge computing (MEC) systems, there are multiple local cache servers caching contents to satisfy the users' requests, instead of letting the users download via the remote cloud server. In this paper, a multi-cell content scheduling problem (MCSP) in MEC systems is considered. Taking into account jointly the freshness of the cached contents and the traffic data costs, we study how to schedule content updates along time in a multi-cell setting. Different from single-cell scenarios, a user may have multiple candidate local cache servers, and thus the caching decisions in all cells must be jointly optimized. We first prove that MCSP is NP-hard, then we formulate MCSP using integer linear programming, by which the optimal scheduling can be obtained for small-scale instances. For problem solving of large scenarios, via a mathematical reformulation, we derive a scalable optimization algorithm based on repeated column generation. Our performance evaluation shows the effectiveness of the proposed algorithm in comparison to an off-the-shelf commercial solver and a popularity-based caching.
Bodily Behaviors in Social Interaction: Novel Annotations and State-of-the-Art Evaluation
Venue: arXiv cs.CV 2022
Year: 2022

Body language is an eye-catching social signal and its automatic analysis can significantly advance artificial intelligence systems to understand and actively participate in social interactions. While computer vision has made impressive progress in low-level tasks like head and body pose estimation, the detection of more subtle behaviors such as gesturing, grooming, or fumbling is not well explored. In this paper we present BBSI, the first set of annotations of complex Bodily Behaviors embedded in continuous Social Interactions in a group setting. Based on previous work in psychology, we manually annotated 26 hours of spontaneous human behavior in the MPIIGroupInteraction dataset with 15 distinct body language classes. We present comprehensive descriptive statistics on the resulting dataset as well as results of annotation quality evaluations. For automatic detection of these behaviors, we adapt the Pyramid Dilated Attention Network (PDAN), a state-of-the-art approach for human action detection. We perform experiments using four variants of spatial-temporal features as input to PDAN: Two-Stream Inflated 3D CNN, Temporal Segment Networks, Temporal Shift Module and Swin Transformer. Results are promising and indicate a great room for improvement in this difficult task. Representing a key piece in the puzzle towards automatic understanding of social behavior, BBSI is fully available to the research community.
Repeated Environment Inference for Invariant Learning
Venue: arXiv cs.LG 2022
Year: 2022

We study the problem of invariant learning when the environment labels are unknown. We focus on the invariant representation notion when the Bayes optimal conditional label distribution is the same across different environments. Previous work conducts Environment Inference (EI) by maximizing the penalty term from Invariant Risk Minimization (IRM) framework. The EI step uses a reference model which focuses on spurious correlations to efficiently reach a good environment partition. However, it is not clear how to find such a reference model. In this work, we propose to repeat the EI process and retrain an ERM model on the \textit{majority} environment inferred by the previous EI step. Under mild assumptions, we find that this iterative process helps learn a representation capturing the spurious correlation better than the single step. This results in better Environment Inference and better Invariant Learning. We show that this method outperforms baselines on both synthetic and real-world datasets.
Safe Model Predictive Control Approach for Non-holonomic Mobile Robots
Venue: arXiv cs.RO 2022
Year: 2022

We design an model predictive control (MPC) approach for planning and control of non-holonomic mobile robots. Linearizing the system dynamics around the pre-computed reference trajectory gives a time-varying LQ MPC problem. We analytically show that by specially designing the MPC controller, the time-varying, linearized system can yield asymptotic stability around the origin in the tracking task. We further propose two obstacle avoidance methods. We show that by defining linearized constraint in velocity-space and explicitly coupling the two control inputs based on current state, our second method directly accounts for the non-holonomic property of the system and therefore alleviates infeasibility of the optimization problems. Simulation results suggest that regarding both static and dynamic obstacle avoidance, the planned trajectories by our LQ MPC approach are comparably smooth and effective as solving non-linear programming (NLP) problems, but in a more efficient way.
A Guide to Image and Video based Small Object Detection using Deep Learning : Case Study of Maritime Surveillance
Venue: arXiv cs.CV 2022
Year: 2022

Small object detection (SOD) in optical images and videos is a challenging problem that even state-of-the-art generic object detection methods fail to accurately localize and identify such objects. Typically, small objects appear in real-world due to large camera-object distance. Because small objects occupy only a small area in the input image (e.g., less than 10%), the information extracted from such a small area is not always rich enough to support decision making. Multidisciplinary strategies are being developed by researchers working at the interface of deep learning and computer vision to enhance the performance of SOD deep learning based methods. In this paper, we provide a comprehensive review of over 160 research papers published between 2017 and 2022 in order to survey this growing subject. This paper summarizes the existing literature and provide a taxonomy that illustrates the broad picture of current research. We investigate how to improve the performance of small object detection in maritime environments, where increasing performance is critical. By establishing a connection between generic and maritime SOD research, future directions have been identified. In addition, the popular datasets that have been used for SOD for generic and maritime applications are discussed, and also well-known evaluation metrics for the state-of-the-art methods on some of the datasets are provided.
From Interpretable Filters to Predictions of Convolutional Neural Networks with Explainable Artificial Intelligence
Venue: arXiv cs.LG 2022
Year: 2022

Convolutional neural networks (CNN) are known for their excellent feature extraction capabilities to enable the learning of models from data, yet are used as black boxes. An interpretation of the convolutional filtres and associated features can help to establish an understanding of CNN to distinguish various classes. In this work, we focus on the explainability of a CNN model called as cnnexplain that is used for Covid-19 and non-Covid-19 classification with a focus on the interpretability of features by the convolutional filters, and how these features contribute to classification. Specifically, we have used various explainable artificial intelligence (XAI) methods, such as visualizations, SmoothGrad, Grad-CAM, and LIME to provide interpretation of convolutional filtres, and relevant features, and their role in classification. We have analyzed the explanation of these methods for Covid-19 detection using dry cough spectrograms. Explanation results obtained from the LIME, SmoothGrad, and Grad-CAM highlight important features of different spectrograms and their relevance to classification.
Student Research Abstract: Microservices-based Systems Visualization
Venue: arXiv cs.SE 2022
Year: 2022

The evolution of decentralized microservice-based systems is challenging. These challenges are classified into static and dynamic categories. Regarding the static perspective, documenting and visualizing the fluid application topology is something few have been able to accomplish. Building an architecture map of services design is a complicated task in its interpretation rather than construction. Therefore, the system-centric and up-to-date view became essential for such distributed systems. The dynamic perspective considers the process of investigation and service path detection. Therefore performing root cause analysis is a burdening task; such that tracing data is needed to be put in the right context to facilitate the investigation. Moreover, visualizing these traces over the traditional visualization techniques couldn't be feasible with the large number of microservices involved in the system. This paper proposes a visualization concept for microservices-based systems using the Augmented Reality (AR) technique, which merges these static and dynamic behaviors into a single centric view. In addition, we challenge the difficulty related to tracing and debugging an issue in such distributed systems. This concept is designed to work as a dynamic documentation and traceability platform for these systems. A proof of concept and a research study are implemented to show the viability and success of this proposal.
Recent insights on the Uniqueness Problem of Diffeomorphisms determined by Prescribed Jacobian Determinant and Curl
Venue: arXiv cs.CG 2022
Year: 2022

Variational Principle (VP) forms diffeomorphisms with prescribed Jacobian determinant (JD) and curl. Examples demonstrate that, (i) JD alone can not uniquely determine a diffeomorphism without curl; and (ii) the solutions by VP seem to satisfy properties of a Lie group. Hence, it is conjectured that a unique diffeomorphism can be assured by its JD and curl (Uniqueness Conjecture). In this paper, (1) an observation based on VP is derived that a counter example to the Conjecture, if exists, should satisfy a particular property; (2) from the observation, an experimental strategy is formulated to numerically test whether a given diffeomorphism is a valid counter example to the conjecture; (3) a proof of an intermediate step to the conjecture is provided and referred to as the semi-general case, which argues that, given two diffeomorphisms, $\pmbϕ$ and $\pmbψ$, if they are close to the identity map, $\pmb{id}$, then $\pmbϕ$ is identical $\pmbψ$.
NNSmith: Generating Diverse and Valid Test Cases for Deep Learning Compilers
Venue: arXiv cs.LG 2022
Year: 2022

Deep-learning (DL) compilers such as TVM and TensorRT are increasingly being used to optimize deep neural network (DNN) models to meet performance, resource utilization and other requirements. Bugs in these compilers can result in models whose semantics differ from the original ones, producing incorrect results that corrupt the correctness of downstream applications. However, finding bugs in these compilers is challenging due to their complexity. In this work, we propose a new fuzz testing approach for finding bugs in deep-learning compilers. Our core approach consists of (i) generating diverse yet valid DNN test models that can exercise a large part of the compiler's transformation logic using light-weight operator specifications; (ii) performing gradient-based search to find model inputs that avoid any floating-point exceptional values during model execution, reducing the chance of missed bugs or false alarms; and (iii) using differential testing to identify bugs. We implemented this approach in NNSmith which has found 72 new bugs for TVM, TensorRT, ONNXRuntime, and PyTorch to date. Of these 58 have been confirmed and 51 have been fixed by their respective project maintainers.