OpenRaiser/Intern-Atlas huggingface dataset:
Target Model Agnostic Adversarial Attacks with Query Budgets on Language Understanding Models
Venue: arXiv cs.LG 2021
Year: 2021

Despite significant improvements in natural language understanding models with the advent of models like BERT and XLNet, these neural-network based classifiers are vulnerable to blackbox adversarial attacks, where the attacker is only allowed to query the target model outputs. We add two more realistic restrictions on the attack methods, namely limiting the number of queries allowed (query budget) and crafting attacks that easily transfer across different pre-trained models (transferability), which render previous attack models impractical and ineffective. Here, we propose a target model agnostic adversarial attack method with a high degree of attack transferability across the attacked models. Our empirical studies show that in comparison to baseline methods, our method generates highly transferable adversarial sentences under the restriction of limited query budgets.
Convex Sparse Blind Deconvolution
Venue: arXiv cs.IT 2021
Year: 2021

In the blind deconvolution problem, we observe the convolution of an unknown filter and unknown signal and attempt to reconstruct the filter and signal. The problem seems impossible in general, since there are seemingly many more unknowns than knowns . Nevertheless, this problem arises in many application fields; and empirically, some of these fields have had success using heuristic methods -- even economically very important ones, in wireless communications and oil exploration. Today's fashionable heuristic formulations pose non-convex optimization problems which are then attacked heuristically as well. The fact that blind deconvolution can be solved under some repeatable and naturally-occurring circumstances poses a theoretical puzzle. To bridge the gulf between reported successes and theory's limited understanding, we exhibit a convex optimization problem that -- assuming signal sparsity -- can convert a crude approximation to the true filter into a high-accuracy recovery of the true filter. Our proposed formulation is based on L1 minimization of inverse filter outputs. We give sharp guarantees on performance of the minimizer assuming sparsity of signal, showing that our proposal precisely recovers the true inverse filter, up to shift and rescaling. There is a sparsity/initial accuracy tradeoff: the less accurate the initial approximation, the greater we rely on sparsity to enable exact recovery. To our knowledge this is the first reported tradeoff of this kind. We consider it surprising that this tradeoff is independent of dimension. We also develop finite-$N$ guarantees, for highly accurate reconstruction under $N\geq O(k \log(k) )$ with high probability. We further show stable approximation when the true inverse filter is infinitely long and extend our guarantees to the case where the observations are contaminated by stochastic or adversarial noise.
Schema-Guided Paradigm for Zero-Shot Dialog
Venue: arXiv cs.CL 2021
Year: 2021

Developing mechanisms that flexibly adapt dialog systems to unseen tasks and domains is a major challenge in dialog research. Neural models implicitly memorize task-specific dialog policies from the training data. We posit that this implicit memorization has precluded zero-shot transfer learning. To this end, we leverage the schema-guided paradigm, wherein the task-specific dialog policy is explicitly provided to the model. We introduce the Schema Attention Model (SAM) and improved schema representations for the STAR corpus. SAM obtains significant improvement in zero-shot settings, with a +22 F1 score improvement over prior work. These results validate the feasibility of zero-shot generalizability in dialog. Ablation experiments are also presented to demonstrate the efficacy of SAM.
HistoTransfer: Understanding Transfer Learning for Histopathology
Venue: arXiv cs.CV 2021
Year: 2021

Advancement in digital pathology and artificial intelligence has enabled deep learning-based computer vision techniques for automated disease diagnosis and prognosis. However, WSIs present unique computational and algorithmic challenges. WSIs are gigapixel-sized, making them infeasible to be used directly for training deep neural networks. Hence, for modeling, a two-stage approach is adopted: Patch representations are extracted first, followed by the aggregation for WSI prediction. These approaches require detailed pixel-level annotations for training the patch encoder. However, obtaining these annotations is time-consuming and tedious for medical experts. Transfer learning is used to address this gap and deep learning architectures pre-trained on ImageNet are used for generating patch-level representation. Even though ImageNet differs significantly from histopathology data, pre-trained networks have been shown to perform impressively on histopathology data. Also, progress in self-supervised and multi-task learning coupled with the release of multiple histopathology data has led to the release of histopathology-specific networks. In this work, we compare the performance of features extracted from networks trained on ImageNet and histopathology data. We use an attention pooling network over these extracted features for slide-level aggregation. We investigate if features learned using more complex networks lead to gain in performance. We use a simple top-k sampling approach for fine-tuning framework and study the representation similarity between frozen and fine-tuned networks using Centered Kernel Alignment. Further, to examine if intermediate block representation is better suited for feature extraction and ImageNet architectures are unnecessarily large for histopathology, we truncate the blocks of ResNet18 and DenseNet121 and examine the performance.
RadArnomaly: Protecting Radar Systems from Data Manipulation Attacks
Venue: arXiv cs.CR 2021
Year: 2021

Radar systems are mainly used for tracking aircraft, missiles, satellites, and watercraft. In many cases, information regarding the objects detected by the radar system is sent to, and used by, a peripheral consuming system, such as a missile system or a graphical user interface used by an operator. Those systems process the data stream and make real-time, operational decisions based on the data received. Given this, the reliability and availability of information provided by radar systems has grown in importance. Although the field of cyber security has been continuously evolving, no prior research has focused on anomaly detection in radar systems. In this paper, we present a deep learning-based method for detecting anomalies in radar system data streams. We propose a novel technique which learns the correlation between numerical features and an embedding representation of categorical features in an unsupervised manner. The proposed technique, which allows the detection of malicious manipulation of critical fields in the data stream, is complemented by a timing-interval anomaly detection mechanism proposed for the detection of message dropping attempts. Real radar system data is used to evaluate the proposed method. Our experiments demonstrate the method's high detection accuracy on a variety of data stream manipulation attacks (average detection rate of 88% with 1.59% false alarms) and message dropping attacks (average detection rate of 92% with 2.2% false alarms).
The Impact of Irrational Behaviours in the Optional Prisoner's Dilemma with Game-Environment Feedback
Venue: arXiv cs.GT 2021
Year: 2021

In the optional prisoner's dilemma (OPD), players can choose to cooperate and defect as usual, but can also abstain as a third possible strategy. This strategy models the players' participation in the game and is a relevant aspect in many settings, e.g. social networks or opinion dynamics where abstention is an option during an election. In this paper, we provide a formulation of the OPD where we consider irrational behaviours in the population inspired by prospect theory. Prospect theory has gained increasing popularity in recent times thanks to its ability to capture aspects such as reference dependence or loss aversion which are common in human behaviour. This element is original in our formulation of the game and is incorporated in our framework through pairwise comparison dynamics. Recently, the impact of the environment has been studied in the form of feedback on the population dynamics. Another element of novelty in our work is the extension of the game-environment feedback to the OPD in two forms of dynamics, the replicator and the pairwise comparison. The contribution of this paper is threefold. First, we propose a modelling framework where prospect theory is used to capture irrational behaviours in an evolutionary game with game-environment feedback. Second, we carry out the stability analysis of the system equilibria and discuss the oscillating behaviours arising from the game-environment feedback. Finally, we extend our previous results to the OPD and we discuss the main differences between the model resulting from the replicator dynamics and the one resulting from the pairwise comparison dynamics.
2rd Place Solutions in the HC-STVG track of Person in Context Challenge 2021
Venue: arXiv cs.CV 2021
Year: 2021

In this technical report, we present our solution to localize a spatio-temporal person in an untrimmed video based on a sentence. We achieve the second vIOU(0.30025) in the HC-STVG track of the 3rd Person in Context(PIC) Challenge. Our solution contains three parts: 1) human attributes information is extracted from the sentence, it is helpful to filter out tube proposals in the testing phase and supervise our classifier to learn appearance information in the training phase. 2) we detect humans with YoloV5 and track humans based on the DeepSort framework but replace the original ReID network with FastReID. 3) a visual transformer is used to extract cross-modal representations for localizing a spatio-temporal tube of the target person.
SGE net: Video object detection with squeezed GRU and information entropy map
Venue: arXiv cs.CV 2021
Year: 2021

Recently, deep learning based video object detection has attracted more and more attention. Compared with object detection of static images, video object detection is more challenging due to the motion of objects, while providing rich temporal information. The RNN-based algorithm is an effective way to enhance detection performance in videos with temporal information. However, most studies in this area only focus on accuracy while ignoring the calculation cost and the number of parameters. In this paper, we propose an efficient method that combines channel-reduced convolutional GRU (Squeezed GRU), and Information Entropy map for video object detection (SGE-Net). The experimental results validate the accuracy improvement, computational savings of the Squeezed GRU, and superiority of the information entropy attention mechanism on the classification performance. The mAP has increased by 3.7 contrasted with the baseline, and the number of parameters has decreased from 6.33 million to 0.67 million compared with the standard GRU.
Automated Parking Space Detection Using Convolutional Neural Networks
Venue: arXiv cs.CV 2021
Year: 2021

Finding a parking space nowadays becomes an issue that is not to be neglected, it consumes time and energy. We have used computer vision techniques to infer the state of the parking lot given the data collected from the University of The Witwatersrand. This paper presents an approach for a real-time parking space classification based on Convolutional Neural Networks (CNN) using Caffe and Nvidia DiGITS framework. The training process has been done using DiGITS and the output is a caffemodel used for predictions to detect vacant and occupied parking spots. The system checks a defined area whether a parking spot (bounding boxes defined at initialization of the system) is containing a car or not (occupied or vacant). Those bounding box coordinates are saved from a frame of the video of the parking lot in a JSON format, to be later used by the system for sequential prediction on each parking spot. The system has been trained using the LeNet network with the Nesterov Accelerated Gradient as solver and the AlexNet network with the Stochastic Gradient Descent as solver. We were able to get an accuracy on the validation set of 99\% for both networks. The accuracy on a foreign dataset(PKLot) returned as well 99\%. Those are experimental results based on the training set shows how robust the system can be when the prediction has to take place in a different parking space.
Flexible dual-branched message passing neural network for quantum mechanical property prediction with molecular conformation
Venue: arXiv cs.LG 2021
Year: 2021

A molecule is a complex of heterogeneous components, and the spatial arrangements of these components determine the whole molecular properties and characteristics. With the advent of deep learning in computational chemistry, several studies have focused on how to predict molecular properties based on molecular configurations. Message passing neural network provides an effective framework for capturing molecular geometric features with the perspective of a molecule as a graph. However, most of these studies assumed that all heterogeneous molecular features, such as atomic charge, bond length, or other geometric features always contribute equivalently to the target prediction, regardless of the task type. In this study, we propose a dual-branched neural network for molecular property prediction based on message-passing framework. Our model learns heterogeneous molecular features with different scales, which are trained flexibly according to each prediction target. In addition, we introduce a discrete branch to learn single atom features without local aggregation, apart from message-passing steps. We verify that this novel structure can improve the model performance with faster convergence in most targets. The proposed model outperforms other recent models with sparser representations. Our experimental results indicate that in the chemical property prediction tasks, the diverse chemical nature of targets should be carefully considered for both model performance and generalizability.
RRULES: An improvement of the RULES rule-based classifier
Venue: arXiv cs.LG 2021
Year: 2021

RRULES is presented as an improvement and optimization over RULES, a simple inductive learning algorithm for extracting IF-THEN rules from a set of training examples. RRULES optimizes the algorithm by implementing a more effective mechanism to detect irrelevant rules, at the same time that checks the stopping conditions more often. This results in a more compact rule set containing more general rules which prevent overfitting the training set and obtain a higher test accuracy. Moreover, the results show that RRULES outperforms the original algorithm by reducing the coverage rate up to a factor of 7 while running twice or three times faster consistently over several datasets.
Node Classification Meets Link Prediction on Knowledge Graphs
Venue: arXiv cs.LG 2021
Year: 2021

Node classification and link prediction are widely studied in graph representation learning. While both transductive node classification and link prediction operate over a single input graph, they have so far been studied separately. Node classification models take an input graph with node features and incomplete node labels, and implicitly assume that the graph is relationally complete, i.e., no edges are missing. By contrast, link prediction models are solely motivated by relational incompleteness of the input graphs, and do not typically leverage node features or classes. We propose a unifying perspective and study the problems of (i) transductive node classification over incomplete graphs and (ii) link prediction over graphs with node features, introduce a new dataset for this setting, WikiAlumni, and conduct an extensive benchmarking study.
iNNformant: Boundary Samples as Telltale Watermarks
Venue: arXiv cs.LG 2021
Year: 2021

Boundary samples are special inputs to artificial neural networks crafted to identify the execution environment used for inference by the resulting output label. The paper presents and evaluates algorithms to generate transparent boundary samples. Transparency refers to a small perceptual distortion of the host signal (i.e., a natural input sample). For two established image classifiers, ResNet on FMNIST and CIFAR10, we show that it is possible to generate sets of boundary samples which can identify any of four tested microarchitectures. These sets can be built to not contain any sample with a worse peak signal-to-noise ratio than 70dB. We analyze the relationship between search complexity and resulting transparency.
MexPub: Deep Transfer Learning for Metadata Extraction from German Publications
Venue: arXiv cs.IR 2021
Year: 2021

Extracting metadata from scientific papers can be considered a solved problem in NLP due to the high accuracy of state-of-the-art methods. However, this does not apply to German scientific publications, which have a variety of styles and layouts. In contrast to most of the English scientific publications that follow standard and simple layouts, the order, content, position and size of metadata in German publications vary greatly among publications. This variety makes traditional NLP methods fail to accurately extract metadata from these publications. In this paper, we present a method that extracts metadata from PDF documents with different layouts and styles by viewing the document as an image. We used Mask R-CNN that is trained on COCO dataset and finetuned with PubLayNet dataset that consists of ~200K PDF snapshots with five basic classes (e.g. text, figure, etc). We refine-tuned the model on our proposed synthetic dataset consisting of ~30K article snapshots to extract nine patterns (i.e. author, title, etc). Our synthetic dataset is generated using contents in both languages German and English and a finite set of challenging templates obtained from German publications. Our method achieved an average accuracy of around $90\%$ which validates its capability to accurately extract metadata from a variety of PDF documents with challenging templates.
MoParkeR : Multi-objective Parking Recommendation
Venue: arXiv cs.IR 2021
Year: 2021

Existing parking recommendation solutions mainly focus on finding and suggesting parking spaces based on the unoccupied options only. However, there are other factors associated with parking spaces that can influence someone's choice of parking such as fare, parking rule, walking distance to destination, travel time, likelihood to be unoccupied at a given time. More importantly, these factors may change over time and conflict with each other which makes the recommendations produced by current parking recommender systems ineffective. In this paper, we propose a novel problem called multi-objective parking recommendation. We present a solution by designing a multi-objective parking recommendation engine called MoParkeR that considers various conflicting factors together. Specifically, we utilise a non-dominated sorting technique to calculate a set of Pareto-optimal solutions, consisting of recommended trade-off parking spots. We conduct extensive experiments using two real-world datasets to show the applicability of our multi-objective recommendation methodology.
Do peers share the same criteria for assessing grant applications?
Venue: arXiv cs.SI 2021
Year: 2021

This study examines a basic assumption of peer review, namely, the idea that there is a consensus on evaluation criteria among peers, which is a necessary condition for the reliability of peer judgements. Empirical evidence indicating that there is no consensus or more than one consensus would offer an explanation for the disagreement effect, the low inter-rater reliability consistently observed in peer review. To investigate this basic assumption, we have surveyed all humanities scholars in Switzerland on 23 grant review criteria. We have employed latent class tree modelling to identify subgroups in which scholars rated criteria similarly (i.e. latent classes) and to explore covariates predicting class membership. We have identified two consensus classes, two consensus-close classes, and a consensus-far class. The consensus classes contain a core consensus (ten criteria related to knowledge gaps, feasibility, rigour, comprehensibility and argumentation, and academic relevance, as well as to the competence and experience of the applicant) and a broad consensus that includes the core consensus plus eight contribution-related criteria, such as originality. These results provide a possible explanation for the disagreement effect. Moreover, the results are consistent with the notion of conservatism, which holds that original research is undervalued in peer review, while other aspects, such as methodology and feasibility, are overweighted. The covariate analysis indicated that age and having tenure increases from the consensus-far to the consensus-close to the consensus classes. This suggests that the more academic experience scholars accumulate, the more their understanding of review criteria conforms to the social norm.
Using Eye Tracker To Evaluate Cockpit Design -- A Flight Simulation Study
Venue: arXiv cs.HC 2021
Year: 2021

This paper investigates applications of eye tracking in transport aircraft design evaluations. Piloted simulations were conducted for a complete flight profile including take off, cruise and landing flight scenario using the transport aircraft flight simulator at CSIR National Aerospace Laboratories. Thirty-one simulation experiments were carried out with three pilots and engineers while recording the ocular parameters and the flight data. Simulations were repeated for high workload conditions like flying with degraded visibility and during stall. Pilots visual scan behaviour and workload levels were analysed using ocular parameters; while comparing with the statistical deviations from the desired flight path. Conditions for fatigue were also recreated through long duration simulations and signatures for the same from the ocular parameters were assessed. Results from the study found correlation between the statistical inferences obtained from the ocular parameters with those obtained from the flight path deviations. The paper also demonstrates an evaluators console that assists the designers or evaluators for better understanding of pilots attentional resource allocation.
Automatically eliminating seam lines with Poisson editing in complex relative radiometric normalization mosaicking scenarios
Venue: arXiv cs.CV 2021
Year: 2021

Relative radiometric normalization (RRN) mosaicking among multiple remote sensing images is crucial for the downstream tasks, including map-making, image recognition, semantic segmentation, and change detection. However, there are often seam lines on the mosaic boundary and radiometric contrast left, especially in complex scenarios, making the appearance of mosaic images unsightly and reducing the accuracy of the latter classification/recognition algorithms. This paper renders a novel automatical approach to eliminate seam lines in complex RRN mosaicking scenarios. It utilizes the histogram matching on the overlap area to alleviate radiometric contrast, Poisson editing to remove the seam lines, and merging procedure to determine the normalization transfer order. Our method can handle the mosaicking seam lines with arbitrary shapes and images with extreme topological relationships (with a small intersection area). These conditions make the main feathering or blending methods, e.g., linear weighted blending and Laplacian pyramid blending, unavailable. In the experiment, our approach visually surpasses the automatic methods without Poisson editing and the manual blurring and feathering method using GIMP software.