OpenRaiser/Intern-Atlas huggingface dataset:
LMVE at SemEval-2020 Task 4: Commonsense Validation and Explanation using Pretraining Language Model
Venue: arXiv cs.CL 2020
Year: 2020

This paper describes our submission to subtask a and b of SemEval-2020 Task 4. For subtask a, we use a ALBERT based model with improved input form to pick out the common sense statement from two statement candidates. For subtask b, we use a multiple choice model enhanced by hint sentence mechanism to select the reason from given options about why a statement is against common sense. Besides, we propose a novel transfer learning strategy between subtasks which help improve the performance. The accuracy scores of our system are 95.6 / 94.9 on official test set and rank 7$^{th}$ / 2$^{nd}$ on Post-Evaluation leaderboard.
A study of multistage interconnection networks operating with wormhole routing and equipped with multi-lane storage
Venue: arXiv cs.NI 2020
Year: 2020

Multistage interconnection networks (MINs) provide critical communication resources between network components with an attractive cost/performance relation. In this paper, a novel architecture for a MIN is proposed. In contrast to other existing networks, this new architecture operates via wormhole routing using a multi-lane equivalent-weighted fair queuing system. Associating the lane storage with each physical channel for movement of the flits allows for high levels of packet flow and makes the system more robust. The resulting flit scheduling schema has been studied thoroughly, and the results are presented in this manuscript. In addition to performance metrics, additional factors such as complexity, cost and reliability are investigated. Since a basic aim of the design of MINs is to achieve a good data flow control mechanism, this proposal is an extremely effective and robust solution. The rationale behind this innovative scheme is to introduce a more efficient network technology, thus providing a better quality of service (i.e. streaming media vs. file transfer).
Learning from Failure: Training Debiased Classifier from Biased Classifier
Venue: arXiv cs.LG 2020
Year: 2020

Neural networks often learn to make predictions that overly rely on spurious correlation existing in the dataset, which causes the model to be biased. While previous work tackles this issue by using explicit labeling on the spuriously correlated attributes or presuming a particular bias type, we instead utilize a cheaper, yet generic form of human knowledge, which can be widely applicable to various types of bias. We first observe that neural networks learn to rely on the spurious correlation only when it is "easier" to learn than the desired knowledge, and such reliance is most prominent during the early phase of training. Based on the observations, we propose a failure-based debiasing scheme by training a pair of neural networks simultaneously. Our main idea is twofold; (a) we intentionally train the first network to be biased by repeatedly amplifying its "prejudice", and (b) we debias the training of the second network by focusing on samples that go against the prejudice of the biased network in (a). Extensive experiments demonstrate that our method significantly improves the training of the network against various types of biases in both synthetic and real-world datasets. Surprisingly, our framework even occasionally outperforms the debiasing methods requiring explicit supervision of the spuriously correlated attributes.
Adversarial Risk Analysis (Overview)
Venue: arXiv cs.GT 2020
Year: 2020

Adversarial risk analysis (ARA) is a relatively new area of research that informs decision-making when facing intelligent opponents and uncertain outcomes. It enables an analyst to express her Bayesian beliefs about an opponent's utilities, capabilities, probabilities and the type of strategic calculation that the opponent is using. Within that framework, the analyst then solves the problem from the perspective of the opponent while placing subjective probability distributions on all unknown quantities. This produces a distribution over the actions of the opponent that permits the analyst to maximize her expected utility. This overview covers conceptual, modeling, computational and applied issues in ARA.
Smart Home, security concerns of IoT
Venue: arXiv cs.CR 2020
Year: 2020

The IoT (Internet of Things) has become widely popular in the domestic environments. People are renewing their homes into smart homes; however, the privacy concerns of owning many Internet connected devices with always-on environmental sensors remain insufficiently addressed. Default and weak passwords, cheap materials and hardware, and unencrypted communication are identified as the principal threats and vulnerabilities of IoT devices. Solutions and countermeasures are also provided: choosing a strong password, strong authentication mechanisms, check online databases of exposed or default credentials to mitigate the first threat; a selection of smart home devices from reputable companies and the implementation of the SDN for the Dos/DDoS threat; and finally IDS, HTTPS protocol and VPN for eavesdropping. The paper concludes dealing with a further challenge, "the lack of technical support", by which an auto-configuration approach should be analysed; this could both ease the installation/maintenance and enhance the security in the self configuration step of Smart Home devices.
A Broad-Coverage Deep Semantic Lexicon for Verbs
Venue: arXiv cs.CL 2020
Year: 2020

Progress on deep language understanding is inhibited by the lack of a broad coverage lexicon that connects linguistic behavior to ontological concepts and axioms. We have developed COLLIE-V, a deep lexical resource for verbs, with the coverage of WordNet and syntactic and semantic details that meet or exceed existing resources. Bootstrapping from a hand-built lexicon and ontology, new ontological concepts and lexical entries, together with semantic role preferences and entailment axioms, are automatically derived by combining multiple constraints from parsing dictionary definitions and examples. We evaluated the accuracy of the technique along a number of different dimensions and were able to obtain high accuracy in deriving new concepts and lexical entries. COLLIE-V is publicly available.
Image Stitching Based on Planar Region Consensus
Venue: arXiv cs.CV 2020
Year: 2020

Image stitching for two images without a global transformation between them is notoriously difficult. In this paper, noticing the importance of planar structure under perspective geometry, we propose a new image stitching method which stitches images by allowing for the alignment of a set of matched dominant planar regions. Clearly different from previous methods resorting to plane segmentation, the key to our approach is to utilize rich semantic information directly from RGB images to extract planar image regions with a deep Convolutional Neural Network (CNN). We specifically design a new module to make fully use of existing semantic segmentation networks to accommodate planar segmentation. To train the network, a dataset for planar region segmentation is contributed. With the planar region knowledge, a set of local transformations can be obtained by constraining matched regions, enabling more precise alignment in the overlapping area. We also use planar knowledge to estimate a transformation field over the whole image. The final mosaic is obtained by a mesh-based optimization framework which maintains high alignment accuracy and relaxes similarity transformation at the same time. Extensive experiments with quantitative comparisons show that our method can deal with different situations and outperforms the state-of-the-arts on challenging scenes.
On $B_1$-EPG and EPT graphs
Venue: arXiv cs.DM 2020
Year: 2020

This research contains as a main result the prove that every Chordal $B_1$-EPG graph is simultaneously in the graph classes VPT and EPT. In addition, we describe structures that must be present in any $B_1$-EPG graph which does not admit a Helly-$B_1$-EPG representation. In particular, this paper presents some features of non-trivial families of graphs properly contained in Helly-$B_1$ EPG, namely Bipartite, Block, Cactus and Line of Bipartite graphs.
Building Reservoir Computing Hardware Using Low Energy-Barrier Magnetics
Venue: arXiv cs.ET 2020
Year: 2020

Biologically inspired recurrent neural networks, such as reservoir computers are of interest in designing spatio-temporal data processors from a hardware point of view due to the simple learning scheme and deep connections to Kalman filters. In this work we discuss using in-depth simulation studies a way to construct hardware reservoir computers using an analog stochastic neuron cell built from a low energy-barrier magnet based magnetic tunnel junction and a few transistors. This allows us to implement a physical embodiment of the mathematical model of reservoir computers. Compact implementation of reservoir computers using such devices may enable building compact, energy-efficient signal processors for standalone or in-situ machine cognition in edge devices.
Certifying Decision Trees Against Evasion Attacks by Program Analysis
Venue: arXiv cs.LG 2020
Year: 2020

Machine learning has proved invaluable for a range of different tasks, yet it also proved vulnerable to evasion attacks, i.e., maliciously crafted perturbations of input data designed to force mispredictions. In this paper we propose a novel technique to verify the security of decision tree models against evasion attacks with respect to an expressive threat model, where the attacker can be represented by an arbitrary imperative program. Our approach exploits the interpretability property of decision trees to transform them into imperative programs, which are amenable for traditional program analysis techniques. By leveraging the abstract interpretation framework, we are able to soundly verify the security guarantees of decision tree models trained over publicly available datasets. Our experiments show that our technique is both precise and efficient, yielding only a minimal number of false positives and scaling up to cases which are intractable for a competitor approach.
Characterizing BigBench queries, Hive, and Spark in multi-cloud environments
Venue: arXiv cs.DC 2020
Year: 2020

BigBench is the new standard (TPCx-BB) for benchmarking and testing Big Data systems. The TPCx-BB specification describes several business use cases -- queries -- which require a broad combination of data extraction techniques including SQL, Map/Reduce (M/R), user code (UDF), and Machine Learning to fulfill them. However, currently, there is no widespread knowledge of the different resource requirements and expected performance of each query, as is the case to more established benchmarks. At the same time, cloud providers currently offer convenient on-demand managed big data clusters (PaaS) with a pay-as-you-go model. In PaaS, analytical engines such as Hive and Spark come ready to use, with a general-purpose configuration and upgrade management. The study characterizes both the BigBench queries and the out-of-the-box performance of Spark and Hive versions in the cloud. At the same time, comparing popular PaaS offerings in terms of reliability, data scalability (1GB to 10TB), versions, and settings from Azure HDinsight, Amazon Web Services EMR, and Google Cloud Dataproc. The query characterization highlights the similarities and differences in Hive an Spark frameworks, and which queries are the most resource consuming according to CPU, memory, and I/O. Scalability results show how there is a need for configuration tuning in most cloud providers as data scale grows, especially with Sparks memory usage. These results can help practitioners to quickly test systems by picking a subset of the queries which stresses each of the categories. At the same time, results show how Hive and Spark compare and what performance can be expected of each in PaaS.
Non-separating spanning trees and out-branchings in digraphsof independence number 2
Venue: arXiv cs.DM 2020
Year: 2020

A subgraph H= (V, F) of a graph G= (V,E) is non-separating if G-F, that is, the graph obtained from G by deleting the edges in F, is connected. Analogously we say that a subdigraph X= (V,B) of a digraph D= (V,A) is non-separating if D-B is strongly connected. We study non-separating spanning trees and out-branchings in digraphs of independence number 2. Our main results are that every 2-arc-strong digraph D of independence number alpha(D) = 2 and minimum in-degree at least 5 and every 2-arc-strong oriented graph with alpha(D) = 2 and minimum in-degree at least 3 has a non-separating out-branching and minimum in-degree 2 is not enough. We also prove a number of other results, including that every 2-arc-strong digraph D with alpha(D)<=2 and at least 14 vertices has a non-separating spanning tree and that every graph G with delta(G)>=4 and alpha(G) = 2 has a non-separating hamiltonian path.
Diagnosis of Coronary Artery Disease Using Artificial Intelligence Based Decision Support System
Venue: arXiv cs.AI 2020
Year: 2020

This research is about the development a fuzzy decision support system for the diagnosis of coronary artery disease based on evidence. The coronary artery disease data sets taken from University California Irvine (UCI) are used. The knowledge base of fuzzy decision support system is taken by using rules extraction method based on Rough Set Theory. The rules then are selected and fuzzified based on information from discretization of numerical attributes. Fuzzy rules weight is proposed using the information from support of extracted rules. UCI heart disease data sets collected from U.S., Switzerland and Hungary, data from Ipoh Specialist Hospital Malaysia are used to verify the proposed system. The results show that the system is able to give the percentage of coronary artery blocking better than cardiologists and angiography. The results of the proposed system were verified and validated by three expert cardiologists and are considered to be more efficient and useful.
Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawords
Venue: arXiv cs.CL 2020
Year: 2020

We present a new release of the Czech-English parallel corpus CzEng 2.0 consisting of over 2 billion words (2 "gigawords") in each language. The corpus contains document-level information and is filtered with several techniques to lower the amount of noise. In addition to the data in the previous version of CzEng, it contains new authentic and also high-quality synthetic parallel data. CzEng is freely available for research and educational purposes.
Labeling of Multilingual Breast MRI Reports
Venue: arXiv cs.CV 2020
Year: 2020

Medical reports are an essential medium in recording a patient's condition throughout a clinical trial. They contain valuable information that can be extracted to generate a large labeled dataset needed for the development of clinical tools. However, the majority of medical reports are stored in an unregularized format, and a trained human annotator (typically a doctor) must manually assess and label each case, resulting in an expensive and time consuming procedure. In this work, we present a framework for developing a multilingual breast MRI report classifier using a custom-built language representation called LAMBR. Our proposed method overcomes practical challenges faced in clinical settings, and we demonstrate improved performance in extracting labels from medical reports when compared with conventional approaches.
Continual Learning in Human Activity Recognition: an Empirical Analysis of Regularization
Venue: arXiv cs.LG 2020
Year: 2020

Given the growing trend of continual learning techniques for deep neural networks focusing on the domain of computer vision, there is a need to identify which of these generalizes well to other tasks such as human activity recognition (HAR). As recent methods have mostly been composed of loss regularization terms and memory replay, we provide a constituent-wise analysis of some prominent task-incremental learning techniques employing these on HAR datasets. We find that most regularization approaches lack substantial effect and provide an intuition of when they fail. Thus, we make the case that the development of continual learning algorithms should be motivated by rather diverse task domains.
Nonlinear Transform Coding
Venue: arXiv cs.IT 2020
Year: 2020

We review a class of methods that can be collected under the name nonlinear transform coding (NTC), which over the past few years have become competitive with the best linear transform codecs for images, and have superseded them in terms of rate--distortion performance under established perceptual quality metrics such as MS-SSIM. We assess the empirical rate--distortion performance of NTC with the help of simple example sources, for which the optimal performance of a vector quantizer is easier to estimate than with natural data sources. To this end, we introduce a novel variant of entropy-constrained vector quantization. We provide an analysis of various forms of stochastic optimization techniques for NTC models; review architectures of transforms based on artificial neural networks, as well as learned entropy models; and provide a direct comparison of a number of methods to parameterize the rate--distortion trade-off of nonlinear transforms, introducing a simplified one.
Text Recognition -- Real World Data and Where to Find Them
Venue: arXiv cs.CV 2020
Year: 2020

We present a method for exploiting weakly annotated images to improve text extraction pipelines. The approach uses an arbitrary end-to-end text recognition system to obtain text region proposals and their, possibly erroneous, transcriptions. The proposed method includes matching of imprecise transcription to weak annotations and edit distance guided neighbourhood search. It produces nearly error-free, localised instances of scene text, which we treat as "pseudo ground truth" (PGT). We apply the method to two weakly-annotated datasets. Training with the extracted PGT consistently improves the accuracy of a state of the art recognition model, by 3.7~\% on average, across different benchmark datasets (image domains) and 24.5~\% on one of the weakly annotated datasets.
Learning Embeddings for Image Clustering: An Empirical Study of Triplet Loss Approaches
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we evaluate two different image clustering objectives, k-means clustering and correlation clustering, in the context of Triplet Loss induced feature space embeddings. Specifically, we train a convolutional neural network to learn discriminative features by optimizing two popular versions of the Triplet Loss in order to study their clustering properties under the assumption of noisy labels. Additionally, we propose a new, simple Triplet Loss formulation, which shows desirable properties with respect to formal clustering objectives and outperforms the existing methods. We evaluate all three Triplet loss formulations for K-means and correlation clustering on the CIFAR-10 image classification dataset.
Learning to learn generative programs with Memoised Wake-Sleep
Venue: arXiv cs.AI 2020
Year: 2020

We study a class of neuro-symbolic generative models in which neural networks are used both for inference and as priors over symbolic, data-generating programs. As generative models, these programs capture compositional structures in a naturally explainable form. To tackle the challenge of performing program induction as an 'inner-loop' to learning, we propose the Memoised Wake-Sleep (MWS) algorithm, which extends Wake Sleep by explicitly storing and reusing the best programs discovered by the inference network throughout training. We use MWS to learn accurate, explainable models in three challenging domains: stroke-based character modelling, cellular automata, and few-shot learning in a novel dataset of real-world string concepts.
Research on Annotation Rules and Recognition Algorithm Based on Phrase Window
Venue: arXiv cs.CL 2020
Year: 2020

At present, most Natural Language Processing technology is based on the results of Word Segmentation for Dependency Parsing, which mainly uses an end-to-end method based on supervised learning. There are two main problems with this method: firstly, the la-beling rules are complex and the data is too difficult to label, the workload of which is large; secondly, the algorithm cannot recognize the multi-granularity and diversity of language components. In order to solve these two problems, we propose labeling rules based on phrase windows, and designed corresponding phrase recognition algorithms. The labeling rule uses phrases as the minimum unit, di-vides sentences into 7 types of nestable phrase types, and marks the grammatical dependencies between phrases. The corresponding algorithm, drawing on the idea of identifying the target area in the image field, can find the start and end positions of various phrases in the sentence, and realize the synchronous recognition of nested phrases and grammatical dependencies. The results of the experiment shows that the labeling rule is convenient and easy to use, and there is no ambiguity; the algorithm is more grammatically multi-granular and diverse than the end-to-end algorithm. Experiments on the CPWD dataset improve the accuracy of the end-to-end method by about 1 point. The corresponding method was applied to the CCL2018 competition, and the first place in the Chinese Metaphor Sentiment Analysis Task.
Discretization-Aware Architecture Search
Venue: arXiv cs.CV 2020
Year: 2020

The search cost of neural architecture search (NAS) has been largely reduced by weight-sharing methods. These methods optimize a super-network with all possible edges and operations, and determine the optimal sub-network by discretization, \textit{i.e.}, pruning off weak candidates. The discretization process, performed on either operations or edges, incurs significant inaccuracy and thus the quality of the final architecture is not guaranteed. This paper presents discretization-aware architecture search (DA\textsuperscript{2}S), with the core idea being adding a loss term to push the super-network towards the configuration of desired topology, so that the accuracy loss brought by discretization is largely alleviated. Experiments on standard image classification benchmarks demonstrate the superiority of our approach, in particular, under imbalanced target network configurations that were not studied before.
Enabling On-Device CNN Training by Self-Supervised Instance Filtering and Error Map Pruning
Venue: arXiv cs.LG 2020
Year: 2020

This work aims to enable on-device training of convolutional neural networks (CNNs) by reducing the computation cost at training time. CNN models are usually trained on high-performance computers and only the trained models are deployed to edge devices. But the statically trained model cannot adapt dynamically in a real environment and may result in low accuracy for new inputs. On-device training by learning from the real-world data after deployment can greatly improve accuracy. However, the high computation cost makes training prohibitive for resource-constrained devices. To tackle this problem, we explore the computational redundancies in training and reduce the computation cost by two complementary approaches: self-supervised early instance filtering on data level and error map pruning on the algorithm level. The early instance filter selects important instances from the input stream to train the network and drops trivial ones. The error map pruning further prunes out insignificant computations when training with the selected instances. Extensive experiments show that the computation cost is substantially reduced without any or with marginal accuracy loss. For example, when training ResNet-110 on CIFAR-10, we achieve 68% computation saving while preserving full accuracy and 75% computation saving with a marginal accuracy loss of 1.3%. Aggressive computation saving of 96% is achieved with less than 0.1% accuracy loss when quantization is integrated into the proposed approaches. Besides, when training LeNet on MNIST, we save 79% computation while boosting accuracy by 0.2%.