OpenRaiser/Intern-Atlas huggingface dataset:
Automata for dynamic answer set solving: Preliminary report
Venue: arXiv cs.AI 2021
Year: 2021

We explore different ways of implementing temporal constraints expressed in an extension of Answer Set Programming (ASP) with language constructs from dynamic logic. Foremost, we investigate how automata can be used for enforcing such constraints. The idea is to transform a dynamic constraint into an automaton expressed in terms of a logic program that enforces the satisfaction of the original constraint. What makes this approach attractive is its independence of time stamps and the potential to detect unsatisfiability. On the one hand, we elaborate upon a transformation of dynamic formulas into alternating automata that relies on meta-programming in ASP. This is the first application of reification applied to theory expressions in gringo. On the other hand, we propose two transformations of dynamic formulas into monadic second-order formulas. These can then be used by off-the-shelf tools to construct the corresponding automata. We contrast both approaches empirically with the one of the temporal ASP solver telingo that directly maps dynamic constraints to logic programs. Since this preliminary study is restricted to dynamic formulas in integrity constraints, its implementations and (empirical) results readily apply to conventional linear dynamic logic, too.
RiWNet: A moving object instance segmentation Network being Robust in adverse Weather conditions
Venue: arXiv cs.CV 2021
Year: 2021

Segmenting each moving object instance in a scene is essential for many applications. But like many other computer vision tasks, this task performs well in optimal weather, but then adverse weather tends to fail. To be robust in weather conditions, the usual way is to train network in data of given weather pattern or to fuse multiple sensors. We focus on a new possibility, that is, to improve its resilience to weather interference through the network's structural design. First, we propose a novel FPN structure called RiWFPN with a progressive top-down interaction and attention refinement module. RiWFPN can directly replace other FPN structures to improve the robustness of the network in non-optimal weather conditions. Then we extend SOLOV2 to capture temporal information in video to learn motion information, and propose a moving object instance segmentation network with RiWFPN called RiWNet. Finally, in order to verify the effect of moving instance segmentation in different weather disturbances, we propose a VKTTI-moving dataset which is a moving instance segmentation dataset based on the VKTTI dataset, taking into account different weather scenes such as rain, fog, sunset, morning as well as overcast. The experiment proves how RiWFPN improves the network's resilience to adverse weather effects compared to other FPN structures. We compare RiWNet to several other state-of-the-art methods in some challenging datasets, and RiWNet shows better performance especially under adverse weather conditions.
A review of Quantum Neural Networks: Methods, Models, Dilemma
Venue: arXiv cs.ET 2021
Year: 2021

The rapid development of quantum computer hardware has laid the hardware foundation for the realization of QNN. Due to quantum properties, QNN shows higher storage capacity and computational efficiency compared to its classical counterparts. This article will review the development of QNN in the past six years from three parts: implementation methods, quantum circuit models, and difficulties faced. Among them, the first part, the implementation method, mainly refers to some underlying algorithms and theoretical frameworks for constructing QNN models, such as VQA. The second part introduces several quantum circuit models of QNN, including QBM, QCVNN and so on. The third part describes some of the main difficult problems currently encountered. In short, this field is still in the exploratory stage, full of magic and practical significance.
On robustness of generative representations against catastrophic forgetting
Venue: arXiv cs.CV 2021
Year: 2021

Catastrophic forgetting of previously learned knowledge while learning new tasks is a widely observed limitation of contemporary neural networks. Although many continual learning methods are proposed to mitigate this drawback, the main question remains unanswered: what is the root cause of catastrophic forgetting? In this work, we aim at answering this question by posing and validating a set of research hypotheses related to the specificity of representations built internally by neural models. More specifically, we design a set of empirical evaluations that compare the robustness of representations in discriminative and generative models against catastrophic forgetting. We observe that representations learned by discriminative models are more prone to catastrophic forgetting than their generative counterparts, which sheds new light on the advantages of developing generative models for continual learning. Finally, our work opens new research pathways and possibilities to adopt generative models in continual learning beyond mere replay mechanisms.
Network Traffic Characteristics of IoT Devices in Smart Homes
Venue: arXiv cs.NI 2021
Year: 2021

Understanding network traffic characteristics of IoT devices plays a critical role in improving both the performance and security of IoT devices, including IoT device identification, classification, and anomaly detection. Although a number of existing research efforts have developed machine-learning based algorithms to help address the challenges in improving the security of IoT devices, none of them have provided detailed studies on the network traffic characteristics of IoT devices. In this paper we collect and analyze the network traffic generated in a typical smart homes environment consisting of a set of common IoT (and non-IoT) devices. We analyze the network traffic characteristics of IoT devices from three complementary aspects: remote network servers and port numbers that IoT devices connect to, flow-level traffic characteristics such as flow duration, and packet-level traffic characteristics such as packet inter-arrival time. Our study provides critical insights into the operational and behavioral characteristics of IoT devices, which can help develop more effective security and performance algorithms for IoT devices.
Pushing Paraphrase Away from Original Sentence: A Multi-Round Paraphrase Generation Approach
Venue: arXiv cs.CL 2021
Year: 2021

In recent years, neural paraphrase generation based on Seq2Seq has achieved superior performance, however, the generated paraphrase still has the problem of lack of diversity. In this paper, we focus on improving the diversity between the generated paraphrase and the original sentence, i.e., making generated paraphrase different from the original sentence as much as possible. We propose BTmPG (Back-Translation guided multi-round Paraphrase Generation), which leverages multi-round paraphrase generation to improve diversity and employs back-translation to preserve semantic information. We evaluate BTmPG on two benchmark datasets. Both automatic and human evaluation show BTmPG can improve the diversity of paraphrase while preserving the semantics of the original sentence.
Training Graph Neural Networks by Graphon Estimation
Venue: arXiv cs.LG 2021
Year: 2021

In this work, we propose to train a graph neural network via resampling from a graphon estimate obtained from the underlying network data. More specifically, the graphon or the link probability matrix of the underlying network is first obtained from which a new network will be resampled and used during the training process at each layer. Due to the uncertainty induced from the resampling, it helps mitigate the well-known issue of over-smoothing in a graph neural network (GNN) model. Our framework is general, computationally efficient, and conceptually simple. Another appealing feature of our method is that it requires minimal additional tuning during the training process. Extensive numerical results show that our approach is competitive with and in many cases outperform the other over-smoothing reducing GNN training methods.
Application Checkpoint and Power Study on Large Scale Systems
Venue: arXiv cs.SE 2021
Year: 2021

Power efficiency is critical in high performance computing (HPC) systems. To achieve high power efficiency on application level, it is vital importance to efficiently distribute power used by application checkpoints. In this study, we analyze the relation of application checkpoints and their power consumption. The observations could guide the design of power management.
Utilizing Adversarial Targeted Attacks to Boost Adversarial Robustness
Venue: arXiv cs.CV 2021
Year: 2021

Adversarial attacks have been shown to be highly effective at degrading the performance of deep neural networks (DNNs). The most prominent defense is adversarial training, a method for learning a robust model. Nevertheless, adversarial training does not make DNNs immune to adversarial perturbations. We propose a novel solution by adopting the recently suggested Predictive Normalized Maximum Likelihood. Specifically, our defense performs adversarial targeted attacks according to different hypotheses, where each hypothesis assumes a specific label for the test sample. Then, by comparing the hypothesis probabilities, we predict the label. Our refinement process corresponds to recent findings of the adversarial subspace properties. We extensively evaluate our approach on 16 adversarial attack benchmarks using ResNet-50, WideResNet-28, and a2-layer ConvNet trained with ImageNet, CIFAR10, and MNIST, showing a significant improvement of up to 5.7%, 3.7%, and 0.6% respectively.
Network Modulation Synthesis: New Algorithms for Generating Musical Audio Using Autoencoder Networks
Venue: arXiv cs.SD 2021
Year: 2021

A new framework is presented for generating musical audio using autoencoder neural networks. With the presented framework, called network modulation synthesis, users can create synthesis architectures and use novel generative algorithms to more easily move through the complex latent parameter space of an autoencoder model to create audio. Implementations of the new algorithms are provided for the open-source CANNe synthesizer network, and can be applied to other autoencoder networks for audio synthesis. Spectrograms and time-series encoding analysis demonstrate that the new algorithms provide simple mechanisms for users to generate time-varying parameter combinations, and therefore auditory possibilities, that are difficult to create by generating audio from handcrafted encodings.
Structural Optimization Makes Graph Classification Simpler and Better
Venue: arXiv cs.LG 2021
Year: 2021

In deep neural networks, better results can often be obtained by increasing the complexity of previously developed basic models. However, it is unclear whether there is a way to boost performance by decreasing the complexity of such models. Here, based on an optimization method, we investigate the feasibility of improving graph classification performance while simplifying the model learning process. Inspired by progress in structural information assessment, we optimize the given data sample from graphs to encoding trees. In particular, we minimize the structural entropy of the transformed encoding tree to decode the key structure underlying a graph. This transformation is denoted as structural optimization. Furthermore, we propose a novel feature combination scheme, termed hierarchical reporting, for encoding trees. In this scheme, features are transferred from leaf nodes to root nodes by following the hierarchical structures of encoding trees. We then present an implementation of the scheme in a tree kernel and a convolutional network to perform graph classification. The tree kernel follows label propagation in the Weisfeiler-Lehman (WL) subtree kernel, but it has a lower runtime complexity $O(n)$. The convolutional network is a special implementation of our tree kernel in the deep learning field and is called Encoding Tree Learning (ETL). We empirically validate our tree kernel and convolutional network with several graph classification benchmarks and demonstrate that our methods achieve better performance and lower computational consumption than competing approaches.
Teaching Autoregressive Language Models Complex Tasks By Demonstration
Venue: arXiv cs.CL 2021
Year: 2021

This paper demonstrates that by fine-tuning an autoregressive language model (GPT-Neo) on appropriately structured step-by-step demonstrations, it is possible to teach it to execute a mathematical task that has previously proved difficult for Transformers - longhand modulo operations - with a relatively small number of examples. Specifically, we fine-tune GPT-Neo to solve the numbers__div_remainder task from the DeepMind Mathematics Dataset; Saxton et al. (arXiv:1904.01557) reported below 40% accuracy on this task with 2 million training examples. We show that after fine-tuning on 200 appropriately structured demonstrations of solving long division problems and reporting the remainders, the smallest available GPT-Neo model achieves over 80% accuracy. This is achieved by constructing an appropriate dataset for fine-tuning, with no changes to the learning algorithm. These results suggest that fine-tuning autoregressive language models on small sets of well-crafted demonstrations may be a useful paradigm for enabling individuals without training in machine learning to coax such models to perform some kinds of complex multi-step tasks.
Temporal Shift Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

The function approximators employed by traditional image-based Deep Reinforcement Learning (DRL) algorithms usually lack a temporal learning component and instead focus on learning the spatial component. We propose a technique, Temporal Shift Reinforcement Learning (TSRL), wherein both temporal, as well as spatial components are jointly learned. Moreover, TSRL does not require additional parameters to perform temporal learning. We show that TSRL outperforms the commonly used frame stacking heuristic on both of the Atari environments we test on while beating the SOTA for one of them. This investigation has implications in the robotics as well as sequential decision-making domains.
Spatial Domain Feature Extraction Methods for Unconstrained Handwritten Malayalam Character Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Handwritten character recognition is an active research challenge,especially for Indian scripts. This paper deals with handwritten Malayalam, with a complete set of basic characters, vowel and consonant signs and compound characters that may be present in the script. Spatial domain features suitable for recognition are chosen in this work. For classification, k-NN, SVM and ELM are employed
Gottesman Types for Quantum Programs
Venue: arXiv cs.LO 2021
Year: 2021

The Heisenberg representation of quantum operators provides a powerful technique for reasoning about quantum circuits, albeit those restricted to the common (non-universal) Clifford set H, S and CNOT. The Gottesman-Knill theorem showed that we can use this representation to efficiently simulate Clifford circuits. We show that Gottesman's semantics for quantum programs can be treated as a type system, allowing us to efficiently characterize a common subset of quantum programs. We also show that it can be extended beyond the Clifford set to partially characterize a broad range of programs. We apply these types to reason about separable states and the superdense coding algorithm.
Achieving Deterministic Service in Mobile Edge Computing (MEC) Networks
Venue: arXiv cs.NI 2021
Year: 2021

Mobile edge computing (MEC) is proposed to boost high-efficient and time-sensitive 5G applications. However, the "microburst" may occur even in lightly-loaded scenarios, which leads to the indeterministic service latency (i.e., unpredictable delay or delay variation), hence hindering the deployment of MEC. Deterministic IP networking (DIP) has been proposed that can provide bounds on latency, and high reliability in the large-scale networks. Nevertheless, the direct migration of DIP into the MEC network is non-trivial owing to its original design for the Ethernet with homogeneous devices. Meanwhile, DIP also faces the challenges on the network throughput and scheduling flexibility. In this paper, we delve into the adoption of DIP for the MEC networks and some of the relevant aspects. A deterministic MEC (D-MEC) network is proposed to deliver the deterministic service (i.e., providing the MEC service with bounded service latency). In the D-MEC network, two mechanisms, including the cycle mapping and cycle shifting, are designed to enable: (i) seamless and deterministic transmission with heterogeneous underlaid resources; and (ii) traffic shaping on the edges to improve the resource utilization. We also formulate a joint configuration to maximize the network throughput with deterministic QoS guarantees. Extensive simulations verify that the proposed D-MEC network can achieve a deterministic MEC service, even in the highly-loaded scenarios.
Learning Fine-Grained Motion Embedding for Landscape Animation
Venue: arXiv cs.CV 2021
Year: 2021

In this paper we focus on landscape animation, which aims to generate time-lapse videos from a single landscape image. Motion is crucial for landscape animation as it determines how objects move in videos. Existing methods are able to generate appealing videos by learning motion from real time-lapse videos. However, current methods suffer from inaccurate motion generation, which leads to unrealistic video results. To tackle this problem, we propose a model named FGLA to generate high-quality and realistic videos by learning Fine-Grained motion embedding for Landscape Animation. Our model consists of two parts: (1) a motion encoder which embeds time-lapse motion in a fine-grained way. (2) a motion generator which generates realistic motion to animate input images. To train and evaluate on diverse time-lapse videos, we build the largest high-resolution Time-lapse video dataset with Diverse scenes, namely Time-lapse-D, which includes 16,874 video clips with over 10 million frames. Quantitative and qualitative experimental results demonstrate the superiority of our method. In particular, our method achieves relative improvements by 19% on LIPIS and 5.6% on FVD compared with state-of-the-art methods on our dataset. A user study carried out with 700 human subjects shows that our approach visually outperforms existing methods by a large margin.
GeneAnnotator: A Semi-automatic Annotation Tool for Visual Scene Graph
Venue: arXiv cs.CV 2021
Year: 2021

In this manuscript, we introduce a semi-automatic scene graph annotation tool for images, the GeneAnnotator. This software allows human annotators to describe the existing relationships between participators in the visual scene in the form of directed graphs, hence enabling the learning and reasoning on visual relationships, e.g., image captioning, VQA and scene graph generation, etc. The annotations for certain image datasets could either be merged in a single VG150 data-format file to support most existing models for scene graph learning or transformed into a separated annotation file for each single image to build customized datasets. Moreover, GeneAnnotator provides a rule-based relationship recommending algorithm to reduce the heavy annotation workload. With GeneAnnotator, we propose Traffic Genome, a comprehensive scene graph dataset with 1000 diverse traffic images, which in return validates the effectiveness of the proposed software for scene graph annotation. The project source code, with usage examples and sample data is available at https://github.com/Milomilo0320/A-Semi-automatic-Annotation-Software-for-Scene-Graph, under the Apache open-source license.
Bit Density Based Signal and Jamming Detection in 1-Bit Quantized MIMO Systems
Venue: arXiv cs.IT 2021
Year: 2021

This paper studies the problem of deciding on the absence (i.e., null hypothesis, $\mathcal{H}_0$) or presence (i.e., alternative hypothesis, $\mathcal{H}_1$) of an unknown signal embedded in the received signal in a multiple-input, multiple-output (MIMO) receiver, employing 1-bit quantization. The originality of our solution lies in quantizing the received signal by an adapted 1-bit window comparator, rather than a traditional 1-bit quantizer. This enables us to divide the space of observed binary sequences into two typical sets (w.r.t. the distribution of the no. of 1's in a sequence) asymptotically, where the first set corresponds to $\mathcal{H}_0$ and the second to $\mathcal{H}_1$. As a result, we reduce the detection problem to determining the highly probable set for an observed sequence. Thus, a very low-complexity binary hypothesis detector is proposed and its probability of detection is given. To show the high efficacy of the proposed 1-bit receiver structure, we consider two wireless applications; jamming detection in a massive MIMO system, and probing a non-stationary low-power transmitter in a wireless sensor network (WSN), assuming unknown Rayleigh-fading channels. Compared with an unquantized system employing a chi-square test, it is shown that the performance loss can be roughly as large as $10\%$ in massive MIMO and this gap diminishes as sequence length or/and jamming power increases. For WSN, we show that compared with an unquantized system, the performance gap becomes smaller when the observation interval is extended over a few symbols.
Improving Numerical Reasoning Skills in the Modular Approach for Complex Question Answering on Text
Venue: arXiv cs.CL 2021
Year: 2021

Numerical reasoning skills are essential for complex question answering (CQA) over text. It requires opertaions including counting, comparison, addition and subtraction. A successful approach to CQA on text, Neural Module Networks (NMNs), follows the programmer-interpreter paradigm and leverages specialised modules to perform compositional reasoning. However, the NMNs framework does not consider the relationship between numbers and entities in both questions and paragraphs. We propose effective techniques to improve NMNs' numerical reasoning capabilities by making the interpreter question-aware and capturing the relationship between entities and numbers. On the same subset of the DROP dataset for CQA on text, experimental results show that our additions outperform the original NMNs by 3.0 points for the overall F1 score.