OpenRaiser/Intern-Atlas huggingface dataset:
DIODE: Dilatable Incremental Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

To accommodate rapid changes in the real world, the cognition system of humans is capable of continually learning concepts. On the contrary, conventional deep learning models lack this capability of preserving previously learned knowledge. When a neural network is fine-tuned to learn new tasks, its performance on previously trained tasks will significantly deteriorate. Many recent works on incremental object detection tackle this problem by introducing advanced regularization. Although these methods have shown promising results, the benefits are often short-lived after the first incremental step. Under multi-step incremental learning, the trade-off between old knowledge preserving and new task learning becomes progressively more severe. Thus, the performance of regularization-based incremental object detectors gradually decays for subsequent learning steps. In this paper, we aim to alleviate this performance decay on multi-step incremental detection tasks by proposing a dilatable incremental object detector (DIODE). For the task-shared parameters, our method adaptively penalizes the changes of important weights for previous tasks. At the same time, the structure of the model is dilated or expanded by a limited number of task-specific parameters to promote new task learning. Extensive experiments on PASCAL VOC and COCO datasets demonstrate substantial improvements over the state-of-the-art methods. Notably, compared with the state-of-the-art methods, our method achieves up to 6.0% performance improvement by increasing the number of parameters by just 1.2% for each newly learned task.
3D-SiamRPN: An End-to-End Learning Method for Real-Time 3D Single Object Tracking Using Raw Point Cloud
Venue: arXiv cs.CV 2021
Year: 2021

3D single object tracking is a key issue for autonomous following robot, where the robot should robustly track and accurately localize the target for efficient following. In this paper, we propose a 3D tracking method called 3D-SiamRPN Network to track a single target object by using raw 3D point cloud data. The proposed network consists of two subnetworks. The first subnetwork is feature embedding subnetwork which is used for point cloud feature extraction and fusion. In this subnetwork, we first use PointNet++ to extract features of point cloud from template and search branches. Then, to fuse the information of features in the two branches and obtain their similarity, we propose two cross correlation modules, named Pointcloud-wise and Point-wise respectively. The second subnetwork is region proposal network(RPN), which is used to get the final 3D bounding box of the target object based on the fusion feature from cross correlation modules. In this subnetwork, we utilize the regression and classification branches of a region proposal subnetwork to obtain proposals and scores, thus get the final 3D bounding box of the target object. Experimental results on KITTI dataset show that our method has a competitive performance in both Success and Precision compared to the state-of-the-art methods, and could run in real-time at 20.8 FPS. Additionally, experimental results on H3D dataset demonstrate that our method also has good generalization ability and could achieve good tracking performance in a new scene without re-training.
On minimal representations of shallow ReLU networks
Venue: arXiv cs.LG 2021
Year: 2021

The realization function of a shallow ReLU network is a continuous and piecewise affine function $f:\mathbb R^d\to \mathbb R$, where the domain $\mathbb R^{d}$ is partitioned by a set of $n$ hyperplanes into cells on which $f$ is affine. We show that the minimal representation for $f$ uses either $n$, $n+1$ or $n+2$ neurons and we characterize each of the three cases. In the particular case, where the input layer is one-dimensional, minimal representations always use at most $n+1$ neurons but in all higher dimensional settings there are functions for which $n+2$ neurons are needed. Then we show that the set of minimal networks representing $f$ forms a $C^\infty$-submanifold $M$ and we derive the dimension and the number of connected components of $M$. Additionally, we give a criterion for the hyperplanes that guarantees that all continuous, piecewise affine functions are realization functions of appropriate ReLU networks.
New Constructions of Golay Complementary Pair/Array with Large Zero Correlation Zone
Venue: arXiv cs.IT 2021
Year: 2021

Zero correlation zone (ZCZ) sequences and Golay sequences are two kinds of sequences with different preferable correlation properties. It was shown by Gong \textit{et al.} and Chen \textit{et al.} that some Golay sequences also possess a large ZCZ and are good candidates for pilots in OFDM systems. Known Golay sequences with ZCZ reported in the literature have a limitation in the length which is the form of a power of 2. One objective of this paper is to propose a construction of Golay complementary pairs (GCPs) with new lengths whose periodic autocorrelation of each of the Golay sequences and periodic corss-correlation of the pair displays a zero correlation zone (ZCZ) around the in-phase position. Specifically, the proposed GCPs have length $4N$ (where, $N$ is the length of a GCP) and ZCZ width $N+1$. Another objective of this paper is to extend the construction to two-dimensional Golay complementary array pairs (GCAPs). Interestingly the periodic corss-correlation of the proposed GACPs also have large ZCZs around the in-phase position.
Deep Microlocal Reconstruction for Limited-Angle Tomography
Venue: arXiv cs.LG 2021
Year: 2021

We present a deep learning-based algorithm to jointly solve a reconstruction problem and a wavefront set extraction problem in tomographic imaging. The algorithm is based on a recently developed digital wavefront set extractor as well as the well-known microlocal canonical relation for the Radon transform. We use the wavefront set information about x-ray data to improve the reconstruction by requiring that the underlying neural networks simultaneously extract the correct ground truth wavefront set and ground truth image. As a necessary theoretical step, we identify the digital microlocal canonical relations for deep convolutional residual neural networks. We find strong numerical evidence for the effectiveness of this approach.
Learning from Matured Dumb Teacher for Fine Generalization
Venue: arXiv cs.LG 2021
Year: 2021

The flexibility of decision boundaries in neural networks that are unguided by training data is a well-known problem typically resolved with generalization methods. A surprising result from recent knowledge distillation (KD) literature is that random, untrained, and equally structured teacher networks can also vastly improve generalization performance. It raises the possibility of existence of undiscovered assumptions useful for generalization on an uncertain region. In this paper, we shed light on the assumptions by analyzing decision boundaries and confidence distributions of both simple and KD-based generalization methods. Assuming that a decision boundary exists to represent the most general tendency of distinction on an input sample space (i.e., the simplest hypothesis), we show the various limitations of methods when using the hypothesis. To resolve these limitations, we propose matured dumb teacher based KD, conservatively transferring the hypothesis for generalization of the student without massive destruction of trained information. In practical experiments on feed-forward and convolution neural networks for image classification tasks on MNIST, CIFAR-10, and CIFAR-100 datasets, the proposed method shows stable improvement to the best test performance in the grid search of hyperparameters. The analysis and results imply that the proposed method can provide finer generalization than existing methods.
Communication-free and Parallel Simulation of Neutral Biodiversity Models
Venue: arXiv cs.DC 2021
Year: 2021

We present a novel communication-free algorithm for individual-based probabilistic neutral biodiversity simulations. The algorithm transforms a neutral Moran ecosystem model into an embarrassingly parallel problem by trading off inter-process communication at the cost of some redundant computation. Specifically, by careful design of the random number generator that drives the simulation, we arrange for evolutionary parent-child interactions to be modelled without requiring knowledge of the interaction, its participants, or which processor is performing the computation. Critically, this means that every individual can be simulated entirely independently. The simulation is thus fully reproducible irrespective of the number of processors it is distributed over. With our novel algorithm, a simulation can be (1) split up into independent batch jobs and (2) simulated across any number of heterogeneous machines - all without affecting the simulation result. We use the Rust programming language to build the extensible and statically checked simulation package $\texttt{necsim-rust}$. We evaluate our parallelisation approach by comparing three traditional simulation algorithms against a CPU and GPU implementation of our Independent algorithm. These experiments show that as long as some local state is maintained to cull redundant individuals, our Independent algorithm is as efficient as existing sequential solutions. The GPU implementation further outperforms all algorithms on the CPU by a factor ranging from $\sim 2$ to $\sim 80$, depending on the model parameterisation and the analysis that is performed. Amongst the parallel algorithms we have investigated, our Independent algorithm provides the only non-approximate parallelisation strategy that can scale to large simulation domains.
Syntax Matters! Syntax-Controlled in Text Style Transfer
Venue: arXiv cs.CL 2021
Year: 2021

Existing text style transfer (TST) methods rely on style classifiers to disentangle the text's content and style attributes for text style transfer. While the style classifier plays a critical role in existing TST methods, there is no known investigation on its effect on the TST methods. In this paper, we conduct an empirical study on the limitations of the style classifiers used in existing TST methods. We demonstrate that the existing style classifiers cannot learn sentence syntax effectively and ultimately worsen existing TST models' performance. To address this issue, we propose a novel Syntax-Aware Controllable Generation (SACG) model, which includes a syntax-aware style classifier that ensures learned style latent representations effectively capture the syntax information for TST. Through extensive experiments on two popular TST tasks, we show that our proposed method significantly outperforms the state-of-the-art methods. Our case studies have also demonstrated SACG's ability to generate fluent target-style sentences that preserved the original content.
HAC Explore: Accelerating Exploration with Hierarchical Reinforcement Learning
Venue: arXiv cs.AI 2021
Year: 2021

Sparse rewards and long time horizons remain challenging for reinforcement learning algorithms. Exploration bonuses can help in sparse reward settings by encouraging agents to explore the state space, while hierarchical approaches can assist with long-horizon tasks by decomposing lengthy tasks into shorter subtasks. We propose HAC Explore (HACx), a new method that combines these approaches by integrating the exploration bonus method Random Network Distillation (RND) into the hierarchical approach Hierarchical Actor-Critic (HAC). HACx outperforms either component method on its own, as well as an existing approach to combining hierarchy and exploration, in a set of difficult simulated robotics tasks. HACx is the first RL method to solve a sparse reward, continuous-control task that requires over 1,000 actions.
Page-level Optimization of e-Commerce Item Recommendations
Venue: arXiv cs.IR 2021
Year: 2021

The item details page (IDP) is a web page on an e-commerce website that provides information on a specific product or item listing. Just below the details of the item on this page, the buyer can usually find recommendations for other relevant items. These are typically in the form of a series of modules or carousels, with each module containing a set of recommended items. The selection and ordering of these item recommendation modules are intended to increase discover-ability of relevant items and encourage greater user engagement, while simultaneously showcasing diversity of inventory and satisfying other business objectives. Item recommendation modules on the IDP are often curated and statically configured for all customers, ignoring opportunities for personalization. In this paper, we present a scalable end-to-end production system to optimize the personalized selection and ordering of item recommendation modules on the IDP in real-time by utilizing deep neural networks. Through extensive offline experimentation and online A/B testing, we show that our proposed system achieves significantly higher click-through and conversion rates compared to other existing methods. In our online A/B test, our framework improved click-through rate by 2.48% and purchase-through rate by 7.34% over a static configuration.
Alzheimer's Disease Diagnosis via Deep Factorization Machine Models
Venue: arXiv cs.LG 2021
Year: 2021

The current state-of-the-art deep neural networks (DNNs) for Alzheimer's Disease diagnosis use different biomarker combinations to classify patients, but do not allow extracting knowledge about the interactions of biomarkers. However, to improve our understanding of the disease, it is paramount to extract such knowledge from the learned model. In this paper, we propose a Deep Factorization Machine model that combines the ability of DNNs to learn complex relationships and the ease of interpretability of a linear model. The proposed model has three parts: (i) an embedding layer to deal with sparse categorical data, (ii) a Factorization Machine to efficiently learn pairwise interactions, and (iii) a DNN to implicitly model higher order interactions. In our experiments on data from the Alzheimer's Disease Neuroimaging Initiative, we demonstrate that our proposed model classifies cognitive normal, mild cognitive impaired, and demented patients more accurately than competing models. In addition, we show that valuable knowledge about the interactions among biomarkers can be obtained.
Where Did the Web Archive Go?
Venue: arXiv cs.DL 2021
Year: 2021

To perform a longitudinal investigation of web archives and detecting variations and changes replaying individual archived pages, or mementos, we created a sample of 16,627 mementos from 17 public web archives. Over the course of our 14-month study (November, 2017 - January, 2019), we found that four web archives changed their base URIs and did not leave a machine-readable method of locating their new base URIs, necessitating manual rediscovery. Of the 1,981 mementos in our sample from these four web archives, 537 were impacted: 517 mementos were rediscovered but with changes in their time of archiving (or Memento-Datetime), HTTP status code, or the string comprising their original URI (or URI-R), and 20 of the mementos could not be found at all.
Sophisticated Students in Boston Mechanism and Gale-Shapley Algorithm for School Choice Problem
Venue: arXiv cs.AI 2021
Year: 2021

We present our experimental results of simulating the school choice problem which deals with the assignment of students to schools based on each group's complete preference list for the other group using two algorithms: Boston mechanism and student-proposing Gale-Shapley algorithm. We compare the effects of sophisticated students altering their preference lists with regards to these two algorithms. Our simulation results show that sophisticated students can benefit more in Boston mechanism compared to Gale-Shapley algorithm based on multiple evaluation metrics.
Using Machine Learning to Predict Engineering Technology Students' Success with Computer Aided Design
Venue: arXiv cs.LG 2021
Year: 2021

Computer-aided design (CAD) programs are essential to engineering as they allow for better designs through low-cost iterations. While CAD programs are typically taught to undergraduate students as a job skill, such software can also help students learn engineering concepts. A current limitation of CAD programs (even those that are specifically designed for educational purposes) is that they are not capable of providing automated real-time help to students. To encourage CAD programs to build in assistance to students, we used data generated from students using a free, open source CAD software called Aladdin to demonstrate how student data combined with machine learning techniques can predict how well a particular student will perform in a design task. We challenged students to design a house that consumed zero net energy as part of an introductory engineering technology undergraduate course. Using data from 128 students, along with the scikit-learn Python machine learning library, we tested our models using both total counts of design actions and sequences of design actions as inputs. We found that our models using early design sequence actions are particularly valuable for prediction. Our logistic regression model achieved a >60% chance of predicting if a student would succeed in designing a zero net energy house. Our results suggest that it would be feasible for Aladdin to provide useful feedback to students when they are approximately halfway through their design. Further improvements to these models could lead to earlier predictions and thus provide students feedback sooner to enhance their learning.
An efficient implementation of the Shamir secret sharing scheme
Venue: arXiv cs.IT 2021
Year: 2021

The Shamir secret sharing scheme requires a Maximum Distance Separable (MDS) code, and in its most common implementation, a Reed-Solomon (RS) code is used. In this paper, we observe that the encoding procedure can be made simpler and faster by dropping the MDS condition and specifying the possible symbols that can be shared. In particular, the process can be made even faster by using array codes based on XOR operations instead of RS codes.
Screenline-based Two-step Calibration and its application to an agent-based urban freight simulator
Venue: arXiv cs.MA 2021
Year: 2021

Calibration is an essential process to make an agent-based simulator operational. Especially, the calibration for freight demand is challenging due to the model complexity and the shortage of available freight demand data compared with passenger data. This paper proposes a novel calibration method that relies solely on screenline counts, named Screenline-based Two-step Calibration (SLTC). SLTC consists of two parts: (1) tour-based demand adjustment and (2) model parameter updates. The former generates screenline-based tours by cloning/removing instances of the simulated goods vehicle tours, aiming to minimize the gaps between the observed and the simulated screenline counts. The latter updates the parameters of the commodity flow model which generates inputs to simulate goods vehicle tours. To demonstrate the practicality of the proposed method, we apply it to an agent-based urban freight simulator, SimMobility Freight. The result shows that SLTC allows the simulator to replicate the observed screenline counts with reasonable computational cost for calibration.
Datasets for Studying Generalization from Easy to Hard Examples
Venue: arXiv cs.LG 2021
Year: 2021

We describe new datasets for studying generalization from easy to hard examples.
Bayesian Modelling of Alluvial Diagram Complexity
Venue: arXiv cs.HC 2021
Year: 2021

Alluvial diagrams are a popular technique for visualizing flow and relational data. However, successfully reading and interpreting the data shown in an alluvial diagram is likely influenced by factors such as data volume, complexity, and chart layout. To understand how alluvial diagram consumption is impacted by its visual features, we conduct two crowdsourced user studies with a set of alluvial diagrams of varying complexity, and examine (i) participant performance on analysis tasks, and (ii) the perceived complexity of the charts. Using the study results, we employ Bayesian modelling to predict participant classification of diagram complexity. We find that, while multiple visual features are important in contributing to alluvial diagram complexity, interestingly the importance of features seems to depend on the type of complexity being modeled, i.e. task complexity vs. perceived complexity.
UMFA: A photorealistic style transfer method based on U-Net and multi-layer feature aggregation
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose a photorealistic style transfer network to emphasize the natural effect of photorealistic image stylization. In general, distortion of the image content and lacking of details are two typical issues in the style transfer field. To this end, we design a novel framework employing the U-Net structure to maintain the rich spatial clues, with a multi-layer feature aggregation (MFA) method to simultaneously provide the details obtained by the shallow layers in the stylization processing. In particular, an encoder based on the dense block and a decoder form a symmetrical structure of U-Net are jointly staked to realize an effective feature extraction and image reconstruction. Besides, a transfer module based on MFA and "adaptive instance normalization" (AdaIN) is inserted in the skip connection positions to achieve the stylization. Accordingly, the stylized image possesses the texture of a real photo and preserves rich content details without introducing any mask or post-processing steps. The experimental results on public datasets demonstrate that our method achieves a more faithful structural similarity with a lower style loss, reflecting the effectiveness and merit of our approach.
Digital Twin of a Cloud Data Centre: OpenStack Cluster Visualisation
Venue: arXiv cs.DC 2021
Year: 2021

Data centres in contemporary times are essential as the supply of data increases. Data centres are areas where computing systems are concentrated for facilitating data processing, transfer and storage. At present traditional data centres have moved more towards the cloud model thereby making the processing, storage and harnessing of data more manageable and more accessible via the utility and subscription-based model of computing services. From the administrative point of view, cloud data centres are complex systems, hard to grasp and require large amounts of time to analyse different aspects of the cloud data centre such as maintenance and resource management. For a cloud data centre admin, this could be a challenging problem and a highly time-consuming task. Accordingly, there is a need to improve the useability of cloud data centre monitoring and management tools, and the digital twin could fulfil this need. This paper's primary objective is to construct a digital twin - a 3D visualisation and monitoring tool - of a cloud data centre managed by OpenStack, the well-known open-source cloud computing infrastructure software. To evaluate our proposed tool, we garner feedback on the digital twin's useability compared to the OpenStack dashboard. The input will be received from cloud data centres experts as they test the digital twin and answer various questions in an interview. The study results show that our proposed Digital Twin will help data centre admins better monitor and manage their data centres. It also will facilitate further research and implementation of the digital twin of data centres to improve usability.