OpenRaiser/Intern-Atlas huggingface dataset:
Throughput-Outage Scaling Behaviors for Wireless Single-Hop D2D Caching Networks with Physical Model -- Analysis and Derivations
Venue: arXiv cs.IT 2021
Year: 2021

Throughput-Outage scaling laws for single-hop cache-aided device-to-device (D2D) communications have been extensively investigated under the assumption of the protocol model. However, the corresponding performance under physical models has not been explored; in particular it remains unclear whether link-level power control and scheduling can improve the asymptotic performance. This paper thus investigates the throughput-outage scaling laws of cache-aided single-hop D2D networks considering a general physical channel model. By considering the networks with and without the equal-throughput assumption, we analyze the corresponding outer bounds and provide the achievable performance analysis. Results show that when the equal-throughput assumption is considered, using link-level power control and scheduling cannot improve the scaling laws. On the other hand, when the equal-throughput assumption is not considered, we show that the proposed double time-slot framework with appropriate link-level power control and scheduling can significantly improve the throughput-outage scaling laws, where the fundamental concept is to first distinguish links according to their communication distances, and then enhance the throughput for links with small communication distances.
Innovation in Large-scale agile -- Benefits and Challenges of Hackathons when Hacking from Home
Venue: arXiv cs.SE 2021
Year: 2021

Hackathons are events in which diverse teams work together to explore, and develop solutions, software or even ideas. Hackathons have been recognized not only as public events for hacking, but also as a corporate mechanism for innovation. Hackathons are a way for established companies to achieve increased employee wellbeing as well as being a curator for innovation and developing new products. Sudden transition to the work-from-home mode caused by the COVID-19 pandemic first put many corporate events requiring collocation, such as hackathons, temporarily on hold and then motivated companies to find ways to hold these events virtually. In this paper, we report our findings from investigating hackathons in the context of a large agile company by first exploring the general benefits and challenges of hackathons and then trying to understand how they were affected by the virtual setup. We conducted nine interviews, surveyed 23 employees and analyzed a hackathon demo. We found that hackathons provide both individual and organizational benefits of innovation, personal interests, and acquiring new skills and competences. Several challenges such as added stress due to stopping the regular work, employees fearing not having enough contribution to deliver and potential mismatch between individual and organizational goals were also found. With respect to the virtual setup, we found that virtual hackathons are not diminishing the innovation benefits, however, some negative effect surfaced on the social and networking side.
AI-Ethics by Design. Evaluating Public Perception on the Importance of Ethical Design Principles of AI
Venue: arXiv cs.CY 2021
Year: 2021

Despite the immense societal importance of ethically designing artificial intelligence (AI), little research on the public perceptions of ethical AI principles exists. This becomes even more striking when considering that ethical AI development has the aim to be human-centric and of benefit for the whole society. In this study, we investigate how ethical principles (explainability, fairness, security, accountability, accuracy, privacy, machine autonomy) are weighted in comparison to each other. This is especially important, since simultaneously considering ethical principles is not only costly, but sometimes even impossible, as developers must make specific trade-off decisions. In this paper, we give first answers on the relative importance of ethical principles given a specific use case - the use of AI in tax fraud detection. The results of a large conjoint survey (n=1099) suggest that, by and large, German respondents found the ethical principles equally important. However, subsequent cluster analysis shows that different preference models for ethically designed systems exist among the German population. These clusters substantially differ not only in the preferred attributes, but also in the importance level of the attributes themselves. We further describe how these groups are constituted in terms of sociodemographics as well as opinions on AI. Societal implications as well as design challenges are discussed.
Consistent Two-Flow Network for Tele-Registration of Point Clouds
Venue: arXiv cs.CV 2021
Year: 2021

Rigid registration of partial observations is a fundamental problem in various applied fields. In computer graphics, special attention has been given to the registration between two partial point clouds generated by scanning devices. State-of-the-art registration techniques still struggle when the overlap region between the two point clouds is small, and completely fail if there is no overlap between the scan pairs. In this paper, we present a learning-based technique that alleviates this problem, and allows registration between point clouds, presented in arbitrary poses, and having little or even no overlap, a setting that has been referred to as tele-registration. Our technique is based on a novel neural network design that learns a prior of a class of shapes and can complete a partial shape. The key idea is combining the registration and completion tasks in a way that reinforces each other. In particular, we simultaneously train the registration network and completion network using two coupled flows, one that register-and-complete, and one that complete-and-register, and encourage the two flows to produce a consistent result. We show that, compared with each separate flow, this two-flow training leads to robust and reliable tele-registration, and hence to a better point cloud prediction that completes the registered scans. It is also worth mentioning that each of the components in our neural network outperforms state-of-the-art methods in both completion and registration. We further analyze our network with several ablation studies and demonstrate its performance on a large number of partial point clouds, both synthetic and real-world, that have only small or no overlap.
Curating Covid-19 data in Links
Venue: arXiv cs.DB 2021
Year: 2021

Curated scientific databases play an important role in the scientific endeavour and support is needed for the significant effort that goes into their creation and maintenance. This demonstration and case study illustrate how curation support has been developed in the Links cross-tier programming language, a functional, strongly typed language with language-integrated query and support for temporal databases. The chosen case study uses weekly released Covid-19 fatality figures from the Scottish government which exhibit updates to previously released data. This data allows the capture and query of update provenance in our prototype. This demonstration will highlight the potential for language-integrated support for curation to simplify and streamline prototyping of web-applications in support of scientific databases
DoT: An efficient Double Transformer for NLP tasks with tables
Venue: arXiv cs.CL 2021
Year: 2021

Transformer-based approaches have been successfully used to obtain state-of-the-art accuracy on natural language processing (NLP) tasks with semi-structured tables. These model architectures are typically deep, resulting in slow training and inference, especially for long inputs. To improve efficiency while maintaining a high accuracy, we propose a new architecture, DoT, a double transformer model, that decomposes the problem into two sub-tasks: A shallow pruning transformer that selects the top-K tokens, followed by a deep task-specific transformer that takes as input those K tokens. Additionally, we modify the task-specific attention to incorporate the pruning scores. The two transformers are jointly trained by optimizing the task-specific loss. We run experiments on three benchmarks, including entailment and question-answering. We show that for a small drop of accuracy, DoT improves training and inference time by at least 50%. We also show that the pruning transformer effectively selects relevant tokens enabling the end-to-end model to maintain similar accuracy as slower baseline models. Finally, we analyse the pruning and give some insight into its impact on the task model.
Omnizart: A General Toolbox for Automatic Music Transcription
Venue: arXiv cs.SD 2021
Year: 2021

We present and release Omnizart, a new Python library that provides a streamlined solution to automatic music transcription (AMT). Omnizart encompasses modules that construct the life-cycle of deep learning-based AMT, and is designed for ease of use with a compact command-line interface. To the best of our knowledge, Omnizart is the first transcription toolkit which offers models covering a wide class of instruments ranging from solo, instrument ensembles, percussion instruments to vocal, as well as models for chord recognition and beat/downbeat tracking, two music information retrieval (MIR) tasks highly related to AMT.
A Novel Graph-Theoretic Deep Representation Learning Method for Multi-Label Remote Sensing Image Retrieval
Venue: arXiv cs.CV 2021
Year: 2021

This paper presents a novel graph-theoretic deep representation learning method in the framework of multi-label remote sensing (RS) image retrieval problems. The proposed method aims to extract and exploit multi-label co-occurrence relationships associated to each RS image in the archive. To this end, each training image is initially represented with a graph structure that provides region-based image representation combining both local information and the related spatial organization. Unlike the other graph-based methods, the proposed method contains a novel learning strategy to train a deep neural network for automatically predicting a graph structure of each RS image in the archive. This strategy employs a region representation learning loss function to characterize the image content based on its multi-label co-occurrence relationship. Experimental results show the effectiveness of the proposed method for retrieval problems in RS compared to state-of-the-art deep representation learning methods. The code of the proposed method is publicly available at https://git.tu-berlin.de/rsim/GT-DRL-CBIR .
Exploring the Diversity and Invariance in Yourself for Visual Pre-Training Task
Venue: arXiv cs.CV 2021
Year: 2021

Recently, self-supervised learning methods have achieved remarkable success in visual pre-training task. By simply pulling the different augmented views of each image together or other novel mechanisms, they can learn much unsupervised knowledge and significantly improve the transfer performance of pre-training models. However, these works still cannot avoid the representation collapse problem, i.e., they only focus on limited regions or the extracted features on totally different regions inside each image are nearly the same. Generally, this problem makes the pre-training models cannot sufficiently describe the multi-grained information inside images, which further limits the upper bound of their transfer performance. To alleviate this issue, this paper introduces a simple but effective mechanism, called Exploring the Diversity and Invariance in Yourself E-DIY. By simply pushing the most different regions inside each augmented view away, E-DIY can preserve the diversity of extracted region-level features. By pulling the most similar regions from different augmented views of the same image together, E-DIY can ensure the robustness of region-level features. Benefited from the above diversity and invariance exploring mechanism, E-DIY maximally extracts the multi-grained visual information inside each image. Extensive experiments on downstream tasks demonstrate the superiority of our proposed approach, e.g., there are 2.1% improvements compared with the strong baseline BYOL on COCO while fine-tuning Mask R-CNN with the R50-C4 backbone and 1X learning schedule.
A Markov Reward Process-Based Approach to Spatial Interpolation
Venue: arXiv cs.AI 2021
Year: 2021

The interpolation of spatial data can be of tremendous value in various applications, such as forecasting weather from only a few measurements of meteorological or remote sensing data. Existing methods for spatial interpolation, such as variants of kriging and spatial autoregressive models, tend to suffer from at least one of the following limitations: (a) the assumption of stationarity, (b) the assumption of isotropy, and (c) the trade-off between modelling local or global spatial interaction. Addressing these issues in this work, we propose the use of Markov reward processes (MRPs) as a spatial interpolation method, and we introduce three variants thereof: (i) a basic static discount MRP (SD-MRP), (ii) an accurate but mostly theoretical optimised MRP (O-MRP), and (iii) a transferable weight prediction MRP (WP-MRP). All variants of MRP interpolation operate locally, while also implicitly accounting for global spatial relationships in the entire system through recursion. Additionally, O-MRP and WP-MRP no longer assume stationarity and are robust to anisotropy. We evaluated our proposed methods by comparing the mean absolute errors of their interpolated grid cells to those of 7 common baselines, selected from models based on spatial autocorrelation, (spatial) regression, and deep learning. We performed detailed evaluations on two publicly available datasets (local GDP values, and COVID-19 patient trajectory data). The results from these experiments clearly show the competitive advantage of MRP interpolation, which achieved significantly lower errors than the existing methods in 23 out of 40 experimental conditions, or 35 out of 40 when including O-MRP.
Prior-Enhanced Few-Shot Segmentation with Meta-Prototypes
Venue: arXiv cs.CV 2021
Year: 2021

Few-shot segmentation~(FSS) performance has been extensively promoted by introducing episodic training and class-wise prototypes. However, the FSS problem remains challenging due to three limitations: (1) Models are distracted by task-unrelated information; (2) The representation ability of a single prototype is limited; (3) Class-related prototypes ignore the prior knowledge of base classes. We propose the Prior-Enhanced network with Meta-Prototypes to tackle these limitations. The prior-enhanced network leverages the support and query (pseudo-) labels in feature extraction, which guides the model to focus on the task-related features of the foreground objects, and suppress much noise due to the lack of supervised knowledge. Moreover, we introduce multiple meta-prototypes to encode hierarchical features and learn class-agnostic structural information. The hierarchical features help the model highlight the decision boundary and focus on hard pixels, and the structural information learned from base classes is treated as the prior knowledge for novel classes. Experiments show that our method achieves the mean-IoU scores of 60.79% and 41.16% on PASCAL-$5^i$ and COCO-$20^i$, outperforming the state-of-the-art method by 3.49% and 5.64% in the 5-shot setting. Moreover, comparing with 1-shot results, our method promotes 5-shot accuracy by 3.73% and 10.32% on the above two benchmarks. The source code of our method is available at https://github.com/Jarvis73/PEMP.
One4all User Representation for Recommender Systems in E-commerce
Venue: arXiv cs.IR 2021
Year: 2021

General-purpose representation learning through large-scale pre-training has shown promising results in the various machine learning fields. For an e-commerce domain, the objective of general-purpose, i.e., one for all, representations would be efficient applications for extensive downstream tasks such as user profiling, targeting, and recommendation tasks. In this paper, we systematically compare the generalizability of two learning strategies, i.e., transfer learning through the proposed model, ShopperBERT, vs. learning from scratch. ShopperBERT learns nine pretext tasks with 79.2M parameters from 0.8B user behaviors collected over two years to produce user embeddings. As a result, the MLPs that employ our embedding method outperform more complex models trained from scratch for five out of six tasks. Specifically, the pre-trained embeddings have superiority over the task-specific supervised features and the strong baselines, which learn the auxiliary dataset for the cold-start problem. We also show the computational efficiency and embedding visualization of the pre-trained features.
Unsupervised detection of mouse behavioural anomalies using two-stream convolutional autoencoders
Venue: arXiv cs.CV 2021
Year: 2021

This paper explores the application of unsupervised learning to detecting anomalies in mouse video data. The two models presented in this paper are a dual-stream, 3D convolutional autoencoder (with residual connections) and a dual-stream, 2D convolutional autoencoder. The publicly available dataset used here contains twelve videos of single home-caged mice alongside frame-level annotations. Under the pretext that the autoencoder only sees normal events, the video data was handcrafted to treat each behaviour as a pseudo-anomaly thereby eliminating them from the others during training. The results are presented for one conspicuous behaviour (hang) and one inconspicuous behaviour (groom). The performance of these models is compared to a single stream autoencoder and a supervised learning model, which are both based on the custom CAE. Both models are also tested on the CUHK Avenue dataset were found to perform as well as some state-of-the-art architectures.
CoRank: A clustering cum graph ranking approach for extractive summarization
Venue: arXiv cs.SI 2021
Year: 2021

Online information has increased tremendously in today's age of Internet. As a result, the need has arose to extract relevant content from the plethora of available information. Researchers are widely using automatic text summarization techniques for extracting useful and relevant information from voluminous available information, it also enables users to obtain valuable knowledge in a limited period of time with minimal effort. The summary obtained from the automatic text summarization often faces the issues of diversity and information coverage. Promising results are obtained for automatic text summarization by the introduction of new techniques based on graph ranking of sentences, clustering, and optimization. This research work proposes CoRank, a two-stage sentence selection model involving clustering and then ranking of sentences. The initial stage involves clustering of sentences using a novel clustering algorithm, and later selection of salient sentences using CoRank algorithm. The approach aims to cover two objectives: maximum coverage and diversity, which is achieved by the extraction of main topics and sub-topics from the original text. The performance of the CoRank is validated on DUC2001 and DUC 2002 data sets.
A (2+ε)-Approximation Algorithm for Maximum Independent Set of Rectangles
Venue: arXiv cs.CG 2021
Year: 2021

We study the Maximum Independent Set of Rectangles (MISR) problem, where we are given a set of axis-parallel rectangles in the plane and the goal is to select a subset of non-overlapping rectangles of maximum cardinality. In a recent breakthrough, Mitchell [2021] obtained the first constant-factor approximation algorithm for MISR. His algorithm achieves an approximation ratio of 10 and it is based on a dynamic program that intuitively recursively partitions the input plane into special polygons called corner-clipped rectangles (CCRs), without intersecting certain special horizontal line segments called fences. In this paper, we present a $(2+ε)$-approximation algorithm for MISR which is also based on a recursive partitioning scheme. First, we use a partition into a class of axis-parallel polygons with constant complexity each that are more general than CCRs. This allows us to provide an arguably simpler analysis and at the same time already improves the approximation ratio to 6. Then, using a more elaborate charging scheme and a recursive partitioning into general axis-parallel polygons with constant complexity, we improve our approximation ratio to $2+ε$. In particular, we construct a recursive partitioning based on more general fences which can be sequences of up to $O(1/ε)$ line segments each. This partitioning routine and our other new ideas may be useful for future work towards a PTAS for MISR.
Parameterized algorithms for identifying gene co-expression modules via weighted clique decomposition
Venue: arXiv cs.DS 2021
Year: 2021

We present a new combinatorial model for identifying regulatory modules in gene co-expression data using a decomposition into weighted cliques. To capture complex interaction effects, we generalize the previously-studied weighted edge clique partition problem. As a first step, we restrict ourselves to the noise-free setting, and show that the problem is fixed parameter tractable when parameterized by the number of modules (cliques). We present two new algorithms for finding these decompositions, using linear programming and integer partitioning to determine the clique weights. Further, we implement these algorithms in Python and test them on a biologically-inspired synthetic corpus generated using real-world data from transcription factors and a latent variable analysis of co-expression in varying cell types.
SoK: Oracles from the Ground Truth to Market Manipulation
Venue: arXiv cs.CR 2021
Year: 2021

One fundamental limitation of blockchain-based smart contracts is that they execute in a closed environment. Thus, they only have access to data and functionality that is already on the blockchain, or is fed into the blockchain. Any interactions with the real world need to be mediated by a bridge service, which is called an oracle. As decentralized applications mature, oracles are playing an increasingly prominent role. With their evolution comes more attacks, necessitating greater attention to their trust model. In this systemization of knowledge paper (SoK), we dissect the design alternatives for oracles, showcase attacks, and discuss attack mitigation strategies.
An Entropy Regularization Free Mechanism for Policy-based Reinforcement Learning
Venue: arXiv cs.LG 2021
Year: 2021

Policy-based reinforcement learning methods suffer from the policy collapse problem. We find valued-based reinforcement learning methods with ε-greedy mechanism are capable of enjoying three characteristics, Closed-form Diversity, Objective-invariant Exploration and Adaptive Trade-off, which help value-based methods avoid the policy collapse problem. However, there does not exist a parallel mechanism for policy-based methods that achieves all three characteristics. In this paper, we propose an entropy regularization free mechanism that is designed for policy-based methods, which achieves Closed-form Diversity, Objective-invariant Exploration and Adaptive Trade-off. Our experiments show that our mechanism is super sample-efficient for policy-based methods and boosts a policy-based baseline to a new State-Of-The-Art on Arcade Learning Environment.
Parallel Polynomial Permanent Mod Powers of 2 and Shortest Disjoint Cycles
Venue: arXiv cs.CC 2021
Year: 2021

We present a parallel algorithm for permanent mod 2^k of a matrix of univariate integer polynomials. It places the problem in ParityL subset of NC^2. This extends the techniques of [Valiant], [Braverman, Kulkarni, Roy] and [Björklund, Husfeldt], and yields a (randomized) parallel algorithm for shortest 2-disjoint paths improving upon the recent result from (randomized) polynomial time. We also recognize the disjoint paths problem as a special case of finding disjoint cycles, and present (randomized) parallel algorithms for finding a shortest cycle and shortest 2-disjoint cycles passing through any given fixed number of vertices or edges.