OpenRaiser/Intern-Atlas huggingface dataset:
Adapting Vehicle Detector to Target Domain by Adversarial Prediction Alignment
Venue: arXiv cs.CV 2021
Year: 2021

While recent advancement of domain adaptation techniques is significant, most of methods only align a feature extractor and do not adapt a classifier to target domain, which would be a cause of performance degradation. We propose novel domain adaptation technique for object detection that aligns prediction output space. In addition to feature alignment, we aligned predictions of locations and class confidences of our vehicle detector for satellite images by adversarial training. The proposed method significantly improved AP score by over 5%, which shows effectivity of our method for object detection tasks in satellite images.
Equivariant bifurcation, quadratic equivariants, and symmetry breaking for the standard representation of $S_n$
Venue: arXiv cs.LG 2021
Year: 2021

Motivated by questions originating from the study of a class of shallow student-teacher neural networks, methods are developed for the analysis of spurious minima in classes of gradient equivariant dynamics related to neural nets. In the symmetric case, methods depend on the generic equivariant bifurcation theory of irreducible representations of the symmetric group on $n$ symbols, $S_n$; in particular, the standard representation of $S_n$. It is shown that spurious minima do not arise from spontaneous symmetry breaking but rather through a complex deformation of the landscape geometry that can be encoded by a generic $S_n$-equivariant bifurcation. We describe minimal models for forced symmetry breaking that give a lower bound on the dynamic complexity involved in the creation of spurious minima when there is no symmetry. Results on generic bifurcation when there are quadratic equivariants are also proved; this work extends and clarifies results of Ihrig & Golubitsky and Chossat, Lauterback & Melbourne on the instability of solutions when there are quadratic equivariants.
Bayesian Nonparametric Modelling for Model-Free Reinforcement Learning in LTE-LAA and Wi-Fi Coexistence
Venue: arXiv cs.LG 2021
Year: 2021

With the arrival of next generation wireless communication, a growing number of new applications like internet of things, autonomous driving systems, and drone are crowding the unlicensed spectrum. Licensed network such as the long-term evolution (LTE) also comes to the unlicensed spectrum for better providing high-capacity contents with low cost. However, LTE was not designed to share resources with others. Previous solutions usually work on fixed scenarios. This work features a Nonparametric Bayesian reinforcement learning algorithm to cope with the coexistence between Wi-Fi and LTE licensed assisted access (LTE-LAA) agents in 5 GHz unlicensed spectrum. The coexistence problem is modeled as a decentralized partially-observable Markov decision process (Dec-POMDP) and Bayesian inference is adopted for policy learning with nonparametric prior to accommodate the uncertainty of policy for different agents. A fairness measure is introduced in the reward function to encourage fair sharing between agents. Variational inference for posterior model approximation is considered to make the algorithm computationally efficient. Simulation results demonstrate that this algorithm can reach high value with compact policy representations in few learning iterations.
The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task
Venue: arXiv cs.CL 2021
Year: 2021

This paper describes the submission of the NiuTrans end-to-end speech translation system for the IWSLT 2021 offline task, which translates from the English audio to German text directly without intermediate transcription. We use the Transformer-based model architecture and enhance it by Conformer, relative position encoding, and stacked acoustic and textual encoding. To augment the training data, the English transcriptions are translated to German translations. Finally, we employ ensemble decoding to integrate the predictions from several models trained with the different datasets. Combining these techniques, we achieve 33.84 BLEU points on the MuST-C En-De test set, which shows the enormous potential of the end-to-end model.
Two new classes of projective two-weight linear codes
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, for an odd prime $p$, several classes of two-weight linear codes over the finite field $\mathbb{F}_p$ are constructed from the defining sets, and then their complete weight distributions are determined by employing character sums. These codes can be suitable for applications in secret sharing schemes. Furthermore, two new classes of projective two-weight codes are obtained, and then two new classes of strongly regular graphs are given.
Size matters? Or not: A/B testing with limited sample in automotive embedded software
Venue: arXiv cs.SE 2021
Year: 2021

A/B testing is gaining attention in the automotive sector as a promising tool to measure causal effects from software changes. Different from the web-facing businesses, where A/B testing has been well-established, the automotive domain often suffers from limited eligible users to participate in online experiments. To address this shortcoming, we present a method for designing balanced control and treatment groups so that sound conclusions can be drawn from experiments with considerably small sample sizes. While the Balance Match Weighted method has been used in other domains such as medicine, this is the first paper to apply and evaluate it in the context of software development. Furthermore, we describe the Balance Match Weighted method in detail and we conduct a case study together with an automotive manufacturer to apply the group design method in a fleet of vehicles. Finally, we present our case study in the automotive software engineering domain, as well as a discussion on the benefits and limitations of the A/B group design method.
A Knowledge graph representation of baseline characteristics for the Dutch proton therapy research registry
Venue: arXiv cs.DB 2021
Year: 2021

Cancer registries collect multisource data and provide valuable information that can lead to unique research opportunities. In the Netherlands, a registry and model-based approach (MBA) are used for the selection of patients that are eligible for proton therapy. We collected baseline characteristics including demographic, clinical, tumour and treatment information. These data were transformed into a machine readable format using the FAIR (Findable, Accessible, Interoperable, Reusable) data principles and resulted in a knowledge graph with baseline characteristics of proton therapy patients. With this approach, we enable the possibility of linking external data sources and optimal flexibility to easily adapt the data structure of the existing knowledge graph to the needs of the clinic.
Metric-Distortion Bounds under Limited Information
Venue: arXiv cs.GT 2021
Year: 2021

In this work we study the metric distortion problem in voting theory under a limited amount of ordinal information. Our primary contribution is threefold. First, we consider mechanisms which perform a sequence of pairwise comparisons between candidates. We show that a widely-popular deterministic mechanism employed in most knockout phases yields distortion $\mathcal{O}(\log m)$ while eliciting only $m-1$ out of $Θ(m^2)$ possible pairwise comparisons, where $m$ represents the number of candidates. Our analysis for this mechanism leverages a powerful technical lemma recently developed by Kempe \cite{DBLP:conf/aaai/000120a}. We also provide a matching lower bound on its distortion. In contrast, we prove that any mechanism which performs fewer than $m-1$ pairwise comparisons is destined to have unbounded distortion. Moreover, we study the power of deterministic mechanisms under incomplete rankings. Most notably, when every agent provides her $k$-top preferences we show an upper bound of $6 m/k + 1$ on the distortion, for any $k \in \{1, 2, \dots, m\}$. Thus, we substantially improve over the previous bound of $12 m/k$ recently established by Kempe \cite{DBLP:conf/aaai/000120a,DBLP:conf/aaai/000120b}, and we come closer to matching the best-known lower bound. Finally, we are concerned with the sample complexity required to ensure near-optimal distortion with high probability. Our main contribution is to show that a random sample of $Θ(m/ε^2)$ voters suffices to guarantee distortion $3 + ε$ with high probability, for any sufficiently small $ε> 0$. This result is based on analyzing the sensitivity of the deterministic mechanism introduced by Gkatzelis, Halpern, and Shah \cite{DBLP:conf/focs/Gkatzelis0020}. Importantly, all of our sample-complexity bounds are distribution-independent.
Intrinsic uncertainties and where to find them
Venue: arXiv cs.LG 2021
Year: 2021

We introduce a framework for uncertainty estimation that both describes and extends many existing methods. We consider typical hyperparameters involved in classical training as random variables and marginalise them out to capture various sources of uncertainty in the parameter space. We investigate which forms and combinations of marginalisation are most useful from a practical point of view on standard benchmarking data sets. Moreover, we discuss how some marginalisations may produce reliable estimates of uncertainty without the need for extensive hyperparameter tuning and/or large-scale ensembling.
Tactile Sensing with a Tendon-Driven Soft Robotic Finger
Venue: arXiv cs.RO 2021
Year: 2021

In this paper, a novel tactile sensing mechanism for soft robotic fingers is proposed. Inspired by the proprioception mechanism found in mammals, the proposed approach infers tactile information from a strain sensor attached on the finger's tendon. We perform experiments to test the tactile sensing capabilities of the proposed structures, and our results indicate this method is capable of palpating texture and stiffness in both abduction and flexion contact. Under systematic cross validation, the proposed system achieved 100% and 99.7% accuracy in texture and stiffness discrimination respectively, which validate the viability of this approach. Furthermore, we use statistics tools to determine the significance of various features extracted for classification.
On the Hardness of Compressing Weights
Venue: arXiv cs.DS 2021
Year: 2021

We investigate computational problems involving large weights through the lens of kernelization, which is a framework of polynomial-time preprocessing aimed at compressing the instance size. Our main focus is the weighted Clique problem, where we are given an edge-weighted graph and the goal is to detect a clique of total weight equal to a prescribed value. We show that the weighted variant, parameterized by the number of vertices $n$, is significantly harder than the unweighted problem by presenting an $O(n^{3 - \varepsilon})$ lower bound on the size of the kernel, under the assumption that NP $\not \subseteq$ coNP/poly. This lower bound is essentially tight: we show that we can reduce the problem to the case with weights bounded by $2^{O(n)}$, which yields a randomized kernel of $O(n^3)$ bits. We generalize these results to the weighted $d$-Uniform Hyperclique problem, Subset Sum, and weighted variants of Boolean Constraint Satisfaction Problems (CSPs). We also study weighted minimization problems and show that weight compression is easier when we only want to preserve the collection of optimal solutions. Namely, we show that for node-weighted Vertex Cover on bipartite graphs it is possible to maintain the set of optimal solutions using integer weights from the range $[1, n]$, but if we want to maintain the ordering of the weights of all inclusion-minimal solutions, then weights as large as $2^{Ω(n)}$ are necessary.
Confidence-based Out-of-Distribution Detection: A Comparative Study and Analysis
Venue: arXiv cs.CV 2021
Year: 2021

Image classification models deployed in the real world may receive inputs outside the intended data distribution. For critical applications such as clinical decision making, it is important that a model can detect such out-of-distribution (OOD) inputs and express its uncertainty. In this work, we assess the capability of various state-of-the-art approaches for confidence-based OOD detection through a comparative study and in-depth analysis. First, we leverage a computer vision benchmark to reproduce and compare multiple OOD detection methods. We then evaluate their capabilities on the challenging task of disease classification using chest X-rays. Our study shows that high performance in a computer vision task does not directly translate to accuracy in a medical imaging task. We analyse factors that affect performance of the methods between the two tasks. Our results provide useful insights for developing the next generation of OOD detection methods.
Point Cloud Registration using Representative Overlapping Points
Venue: arXiv cs.CV 2021
Year: 2021

3D point cloud registration is a fundamental task in robotics and computer vision. Recently, many learning-based point cloud registration methods based on correspondences have emerged. However, these methods heavily rely on such correspondences and meet great challenges with partial overlap. In this paper, we propose ROPNet, a new deep learning model using Representative Overlapping Points with discriminative features for registration that transforms partial-to-partial registration into partial-to-complete registration. Specifically, we propose a context-guided module which uses an encoder to extract global features for predicting point overlap score. To better find representative overlapping points, we use the extracted global features for coarse alignment. Then, we introduce a Transformer to enrich point features and remove non-representative points based on point overlap score and feature matching. A similarity matrix is built in a partial-to-complete mode, and finally, weighted SVD is adopted to estimate a transformation matrix. Extensive experiments over ModelNet40 using noisy and partially overlapping point clouds show that the proposed method outperforms traditional and learning-based methods, achieving state-of-the-art performance. The code is available at https://github.com/zhulf0804/ROPNet.
Meta-Reinforcement Learning for Heuristic Planning
Venue: arXiv cs.AI 2021
Year: 2021

In Meta-Reinforcement Learning (meta-RL) an agent is trained on a set of tasks to prepare for and learn faster in new, unseen, but related tasks. The training tasks are usually hand-crafted to be representative of the expected distribution of test tasks and hence all used in training. We show that given a set of training tasks, learning can be both faster and more effective (leading to better performance in the test tasks), if the training tasks are appropriately selected. We propose a task selection algorithm, Information-Theoretic Task Selection (ITTS), based on information theory, which optimizes the set of tasks used for training in meta-RL, irrespectively of how they are generated. The algorithm establishes which training tasks are both sufficiently relevant for the test tasks, and different enough from one another. We reproduce different meta-RL experiments from the literature and show that ITTS improves the final performance in all of them.
DocSynth: A Layout Guided Approach for Controllable Document Image Synthesis
Venue: arXiv cs.CV 2021
Year: 2021

Despite significant progress on current state-of-the-art image generation models, synthesis of document images containing multiple and complex object layouts is a challenging task. This paper presents a novel approach, called DocSynth, to automatically synthesize document images based on a given layout. In this work, given a spatial layout (bounding boxes with object categories) as a reference by the user, our proposed DocSynth model learns to generate a set of realistic document images consistent with the defined layout. Also, this framework has been adapted to this work as a superior baseline model for creating synthetic document image datasets for augmenting real data during training for document layout analysis tasks. Different sets of learning objectives have been also used to improve the model performance. Quantitatively, we also compare the generated results of our model with real data using standard evaluation metrics. The results highlight that our model can successfully generate realistic and diverse document images with multiple objects. We also present a comprehensive qualitative analysis summary of the different scopes of synthetic image generation tasks. Lastly, to our knowledge this is the first work of its kind.
MAJORITY-3SAT (and Related Problems) in Polynomial Time
Venue: arXiv cs.CC 2021
Year: 2021

Majority-SAT is the problem of determining whether an input $n$-variable formula in conjunctive normal form (CNF) has at least $2^{n-1}$ satisfying assignments. Majority-SAT and related problems have been studied extensively in various AI communities interested in the complexity of probabilistic planning and inference. Although Majority-SAT has been known to be PP-complete for over 40 years, the complexity of a natural variant has remained open: Majority-$k$SAT, where the input CNF formula is restricted to have clause width at most $k$. We prove that for every $k$, Majority-$k$SAT is in P. In fact, for any positive integer $k$ and rational $ρ\in (0,1)$ with bounded denominator, we give an algorithm that can determine whether a given $k$-CNF has at least $ρ\cdot 2^n$ satisfying assignments, in deterministic linear time (whereas the previous best-known algorithm ran in exponential time). Our algorithms have interesting positive implications for counting complexity and the complexity of inference, significantly reducing the known complexities of related problems such as E-MAJ-$k$SAT and MAJ-MAJ-$k$SAT. At the heart of our approach is an efficient method for solving threshold counting problems by extracting sunflowers found in the corresponding set system of a $k$-CNF. We also show that the tractability of Majority-$k$SAT is somewhat fragile. For the closely related GtMajority-SAT problem (where we ask whether a given formula has greater than $2^{n-1}$ satisfying assignments) which is known to be PP-complete, we show that GtMajority-$k$SAT is in P for $k\le 3$, but becomes NP-complete for $k\geq 4$. These results are counterintuitive, because the ``natural'' classifications of these problems would have been PP-completeness, and because there is a stark difference in the complexity of GtMajority-$k$SAT and Majority-$k$SAT for all $k\ge 4$.
Benchmarking (multi)wavelet-based dynamic and static non-uniform grid solvers for flood inundation modelling
Venue: arXiv cs.CE 2021
Year: 2021

This paper explores static non-uniform grid solvers that adapt three raster-based flood models on an optimised non-uniform grid: the second-order discontinuous Galerkin (DG2) model representing the modelled data as piecewise-planar fields, the first-order finite volume (FV1) model using piecewise-constant fields, and the local inertial (ACC) model only evolving piecewise-constant water depth fields. The optimised grid is generated by applying the multiresolution analysis (MRA) of multiwavelets (MWs) to piecewise-planar representation of raster-formatted topography data, for more sensible grid coarsening based on one user-specified parameter. Two adaptive solvers are also explored that apply the MRA of MWs and of Haar wavelets (HWs) to, respectively, scale and adapt the DG2 (MWDG2) and FV1 (HWFV1) modelled data dynamically in time. The performance of the non-uniform grid and adaptive solvers is assessed in terms of flood depth and extent, velocities, and CPU runtimes, with reference to the raster-based DG2 model predictions on their finest resolution grid. The assessments considered three large-scale flooding scenarios, involving rapid and slow-to-gradual flows. MWDG2 is found to be the most favourable choice when modelling rapid flows, where it excels in capturing small velocity variations. For slow-to-gradual flows, the adaptive solvers deliver less accurate outcomes, and their efficiency can be hampered by overhead costs of the dynamic MRA. Instead, non-uniform DG2 is recommended to capture urban flow interactions more accurately. Non-uniform ACC is 5 times faster to run than non-uniform DG2 but delivers close flooding depth and extent predictions, thus is more attractive for fluvial/pluvial flood simulation over large areas.
Machine Learning for Network-based Intrusion Detection Systems: an Analysis of the CIDDS-001 Dataset
Venue: arXiv cs.CR 2021
Year: 2021

With the increasing amount of reliance on digital data and computer networks by corporations and the public in general, the occurrence of cyber attacks has become a great threat to the normal functioning of our society. Intrusion detection systems seek to address this threat by preemptively detecting attacks in real time while attempting to block them or minimizing their damage. These systems can function in many ways being some of them based on artificial intelligence methods. Datasets containing both normal network traffic and cyber attacks are used for training these algorithms so that they can learn the underlying patterns of network-based data. The CIDDS-001 is one of the most used datasets for network-based intrusion detection research. Regarding this dataset, in the majority of works published so far, the Class label was used for training machine learning algorithms. However, there is another label in the CIDDS-001, AttackType, that seems very promising for this purpose and remains considerably unexplored. This work seeks to make a comparison between two machine learning models, K-Nearest Neighbours and Random Forest, which were trained with both these labels in order to ascertain whether AttackType can produce reliable results in comparison with the Class label.
Predicting Surface Heat Flux on Complex Systems via Conv-LSTM
Venue: arXiv cs.CE 2021
Year: 2021

Existing algorithms with iterations as the principle for 3D inverse heat conduction problems (IHCPs) are usually time-consuming. With the recent advancements in deep learning techniques, it is possible to apply the neural network to compute IHCPs. In this paper, a new framework based on Convolutional-LSTM is introduced to predict the transient heat flux via measured temperature. The inverse heat conduction models concerned in this work have 3D complex structures with non-linear boundary conditions and thermophysical parameters. In order to reach high precision, a forward solver based on the finite element method is utilized to generate sufficient data for training. The fully trained framework can provide accurate predictions efficiently once the measured temperature and models are acquired. It is believed that the proposed framework offers a new pattern for real-time heat flux inversion.
Generalization Error Analysis of Neural networks with Gradient Based Regularization
Venue: arXiv cs.LG 2021
Year: 2021

We study gradient-based regularization methods for neural networks. We mainly focus on two regularization methods: the total variation and the Tikhonov regularization. Applying these methods is equivalent to using neural networks to solve some partial differential equations, mostly in high dimensions in practical applications. In this work, we introduce a general framework to analyze the generalization error of regularized networks. The error estimate relies on two assumptions on the approximation error and the quadrature error. Moreover, we conduct some experiments on the image classification tasks to show that gradient-based methods can significantly improve the generalization ability and adversarial robustness of neural networks. A graphical extension of the gradient-based methods are also considered in the experiments.