OpenRaiser/Intern-Atlas huggingface dataset:
A Learnable Self-supervised Task for Unsupervised Domain Adaptation on Point Clouds
Venue: arXiv cs.CV 2021
Year: 2021

Deep neural networks have achieved promising performance in supervised point cloud applications, but manual annotation is extremely expensive and time-consuming in supervised learning schemes. Unsupervised domain adaptation (UDA) addresses this problem by training a model with only labeled data in the source domain but making the model generalize well in the target domain. Existing studies show that self-supervised learning using both source and target domain data can help improve the adaptability of trained models, but they all rely on hand-crafted designs of the self-supervised tasks. In this paper, we propose a learnable self-supervised task and integrate it into a self-supervision-based point cloud UDA architecture. Specifically, we propose a learnable nonlinear transformation that transforms a part of a point cloud to generate abundant and complicated point clouds while retaining the original semantic information, and the proposed self-supervised task is to reconstruct the original point cloud from the transformed ones. In the UDA architecture, an encoder is shared between the networks for the self-supervised task and the main task of point cloud classification or segmentation, so that the encoder can be trained to extract features suitable for both the source and the target domain data. Experiments on PointDA-10 and PointSegDA datasets show that the proposed method achieves new state-of-the-art performance on both classification and segmentation tasks of point cloud UDA. Code will be made publicly available.
How Asymmetry Helps Buffer Management: Achieving Optimal Tail Size in Cup Games
Venue: arXiv cs.DS 2021
Year: 2021

The cup game on $n$ cups is a multi-step game with two players, a filler and an emptier. At each step, the filler distributes $1$ unit of water among the cups, and then the emptier selects a single cup to remove (up to) $1$ unit of water from. There are several objective functions that the emptier might wish to minimize. One of the strongest guarantees would be to minimize tail size, which is defined to be the number of cups with fill $2$ or greater. A simple lower-bound construction shows that the optimal tail size for deterministic emptying algorithms is $Θ(n)$, however. We present a simple randomized emptying algorithm that achieves tail size $\tilde{O}(\log n)$ with high probability in $n$ for $\operatorname{poly} n$ steps. Moreover, we show that this is tight up to doubly logarithmic factors. We also extend our results to the multi-processor cup game, achieving tail size $\tilde{O}(\log n + p)$ on $p$ processors with high probability in $n$. We show that the dependence on $p$ is near optimal for any emptying algorithm that achieves polynomial-bounded backlog. A natural question is whether our results can be extended to give unending guarantees, which apply to arbitrarily long games. We give a lower bound construction showing that no monotone memoryless emptying algorithm can achieve an unending guarantee on either tail size or the related objective function of backlog. On the other hand, we show that even a very small (i.e., $1 / \operatorname{poly} n$) amount of resource augmentation is sufficient to overcome this barrier.
Accelerating science with human versus alien artificial intelligences
Venue: arXiv cs.AI 2021
Year: 2021

Data-driven artificial intelligence models fed with published scientific findings have been used to create powerful prediction engines for scientific and technological advance, such as the discovery of novel materials with desired properties and the targeted invention of new therapies and vaccines. These AI approaches typically ignore the distribution of human prediction engines -- scientists and inventor -- who continuously alter the landscape of discovery and invention. As a result, AI hypotheses are designed to substitute for human experts, failing to complement them for punctuated collective advance. Here we show that incorporating the distribution of human expertise into self-supervised models by training on inferences cognitively available to experts dramatically improves AI prediction of future human discoveries and inventions. Including expert-awareness into models that propose (a) valuable energy-relevant materials increases the precision of materials predictions by ~100%, (b) repurposing thousands of drugs to treat new diseases increases precision by 43%, and (c) COVID-19 vaccine candidates examined in clinical trials by 260%. These models succeed by predicting human predictions and the scientists who will make them. By tuning AI to avoid the crowd, however, it generates scientifically promising "alien" hypotheses unlikely to be imagined or pursued without intervention, not only accelerating but punctuating scientific advance. By identifying and correcting for collective human bias, these models also suggest opportunities to improve human prediction by reformulating science education for discovery.
Online Machine Learning Techniques for Coq: A Comparison
Venue: arXiv cs.LO 2021
Year: 2021

We present a comparison of several online machine learning techniques for tactical learning and proving in the Coq proof assistant. This work builds on top of Tactician, a plugin for Coq that learns from proofs written by the user to synthesize new proofs. Learning happens in an online manner, meaning that Tactician's machine learning model is updated immediately every time the user performs a step in an interactive proof. This has important advantages compared to the more studied offline learning systems: (1) it provides the user with a seamless, interactive experience with Tactician and, (2) it takes advantage of locality of proof similarity, which means that proofs similar to the current proof are likely to be found close by. We implement two online methods, namely approximate k-nearest neighbors based on locality sensitive hashing forests and random decision forests. Additionally, we conduct experiments with gradient boosted trees in an offline setting using XGBoost. We compare the relative performance of Tactician using these three learning methods on Coq's standard library.
Information Rate Optimization for Non-Regenerative MIMO Relay Networks with a Direct Link
Venue: arXiv cs.IT 2021
Year: 2021

We consider the optimization of a two-hop relay network based on an amplify-and-forward Multiple-Input Multiple-Output (MIMO) relay. The relay is assumed to derive the output signal by a Relay Transform Matrix (RTM) applied to the input signal. Assuming perfect channel state information about the network at the relay, the RTM is optimized according to two different criteria: {\bf\em i)} network capacity; {\bf\em ii)} network capacity based on Orthogonal Space--Time Block Codes. The two assumptions have been addressed in part in the literature. The optimization problem is reduced to a manageable convex form, whose KKT equations are explicitly solved. Then, a parametric solution is given, which yields the power constraint and the capacity achieved with uncorrelated transmitted data as functions of a positive indeterminate. The solution for a given average power constraint at the relay is amenable to a \emph{water-filling-like} algorithm, and extends earlier literature results addressing the case without the direct link. Simulation results are reported concerning a Rayleigh relay network and the role of the direct link SNR is precisely assessed.
Neural Camera Simulators
Venue: arXiv cs.CV 2021
Year: 2021

We present a controllable camera simulator based on deep neural networks to synthesize raw image data under different camera settings, including exposure time, ISO, and aperture. The proposed simulator includes an exposure module that utilizes the principle of modern lens designs for correcting the luminance level. It also contains a noise module using the noise level function and an aperture module with adaptive attention to simulate the side effects on noise and defocus blur. To facilitate the learning of a simulator model, we collect a dataset of the 10,000 raw images of 450 scenes with different exposure settings. Quantitative experiments and qualitative comparisons show that our approach outperforms relevant baselines in raw data synthesize on multiple cameras. Furthermore, the camera simulator enables various applications, including large-aperture enhancement, HDR, auto exposure, and data augmentation for training local feature detectors. Our work represents the first attempt to simulate a camera sensor's behavior leveraging both the advantage of traditional raw sensor features and the power of data-driven deep learning.
Robust Classification from Noisy Labels: Integrating Additional Knowledge for Chest Radiography Abnormality Assessment
Venue: arXiv cs.CV 2021
Year: 2021

Chest radiography is the most common radiographic examination performed in daily clinical practice for the detection of various heart and lung abnormalities. The large amount of data to be read and reported, with more than 100 studies per day for a single radiologist, poses a challenge in consistently maintaining high interpretation accuracy. The introduction of large-scale public datasets has led to a series of novel systems for automated abnormality classification. However, the labels of these datasets were obtained using natural language processed medical reports, yielding a large degree of label noise that can impact the performance. In this study, we propose novel training strategies that handle label noise from such suboptimal data. Prior label probabilities were measured on a subset of training data re-read by 4 board-certified radiologists and were used during training to increase the robustness of the training model to the label noise. Furthermore, we exploit the high comorbidity of abnormalities observed in chest radiography and incorporate this information to further reduce the impact of label noise. Additionally, anatomical knowledge is incorporated by training the system to predict lung and heart segmentation, as well as spatial knowledge labels. To deal with multiple datasets and images derived from various scanners that apply different post-processing techniques, we introduce a novel image normalization strategy. Experiments were performed on an extensive collection of 297,541 chest radiographs from 86,876 patients, leading to a state-of-the-art performance level for 17 abnormalities from 2 datasets. With an average AUC score of 0.880 across all abnormalities, our proposed training strategies can be used to significantly improve performance scores.
Intra-Class Uncertainty Loss Function for Classification
Venue: arXiv cs.CV 2021
Year: 2021

Most classification models can be considered as the process of matching templates. However, when intra-class uncertainty/variability is not considered, especially for datasets containing unbalanced classes, this may lead to classification errors. To address this issue, we propose a loss function with intra-class uncertainty following Gaussian distribution. Specifically, in our framework, the features extracted by deep networks of each class are characterized by independent Gaussian distribution. The parameters of distribution are learned with a likelihood regularization along with other network parameters. The means of the Gaussian play a similar role as the center anchor in existing methods, and the variance describes the uncertainty of different classes. In addition, similar to the inter-class margin in traditional loss functions, we introduce a margin to intra-class uncertainty to make each cluster more compact and reduce the imbalance of feature distribution from different categories. Based on MNIST, CIFAR, ImageNet, and Long-tailed CIFAR analyses, the proposed approach shows improved classification performance, through learning a better class representation.
Machine learning and deep learning
Venue: arXiv cs.AI 2021
Year: 2021

Today, intelligent systems that offer artificial intelligence capabilities often rely on machine learning. Machine learning describes the capacity of systems to learn from problem-specific training data to automate the process of analytical model building and solve associated tasks. Deep learning is a machine learning concept based on artificial neural networks. For many applications, deep learning models outperform shallow machine learning models and traditional data analysis approaches. In this article, we summarize the fundamentals of machine learning and deep learning to generate a broader understanding of the methodical underpinning of current intelligent systems. In particular, we provide a conceptual distinction between relevant terms and concepts, explain the process of automated analytical model building through machine learning and deep learning, and discuss the challenges that arise when implementing such intelligent systems in the field of electronic markets and networked business. These naturally go beyond technological aspects and highlight issues in human-machine interaction and artificial intelligence servitization.
Combining exogenous and endogenous signals with a semi-supervised co-attention network for early detection of COVID-19 fake tweets
Venue: arXiv cs.CL 2021
Year: 2021

Fake tweets are observed to be ever-increasing, demanding immediate countermeasures to combat their spread. During COVID-19, tweets with misinformation should be flagged and neutralized in their early stages to mitigate the damages. Most of the existing methods for early detection of fake news assume to have enough propagation information for large labeled tweets -- which may not be an ideal setting for cases like COVID-19 where both aspects are largely absent. In this work, we present ENDEMIC, a novel early detection model which leverages exogenous and endogenous signals related to tweets, while learning on limited labeled data. We first develop a novel dataset, called CTF for early COVID-19 Twitter fake news, with additional behavioral test sets to validate early detection. We build a heterogeneous graph with follower-followee, user-tweet, and tweet-retweet connections and train a graph embedding model to aggregate propagation information. Graph embeddings and contextual features constitute endogenous, while time-relative web-scraped information constitutes exogenous signals. ENDEMIC is trained in a semi-supervised fashion, overcoming the challenge of limited labeled data. We propose a co-attention mechanism to fuse signal representations optimally. Experimental results on ECTF, PolitiFact, and GossipCop show that ENDEMIC is highly reliable in detecting early fake tweets, outperforming nine state-of-the-art methods significantly.
Feedback Vertex Set on Hamiltonian Graphs
Venue: arXiv cs.CC 2021
Year: 2021

We study the computational complexity of Feedback Vertex Set on subclasses of Hamiltonian graphs. In particular, we consider Hamiltonian graphs that are regular or are planar and regular. Moreover, we study the less known class of $p$-Hamiltonian-ordered graphs, which are graphs that admit for any $p$-tuple of vertices a Hamiltonian cycle visiting them in the order given by the tuple. We prove that Feedback Vertex Set remains NP-hard in these restricted cases, even if a Hamiltonian cycle is additionally given as part of the input.
RPSRNet: End-to-End Trainable Rigid Point Set Registration Network using Barnes-Hut $2^D$-Tree Representation
Venue: arXiv cs.CV 2021
Year: 2021

We propose RPSRNet - a novel end-to-end trainable deep neural network for rigid point set registration. For this task, we use a novel $2^D$-tree representation for the input point sets and a hierarchical deep feature embedding in the neural network. An iterative transformation refinement module in our network boosts the feature matching accuracy in the intermediate stages. We achieve an inference speed of 12-15ms to register a pair of input point clouds as large as 250K. Extensive evaluation on (i) KITTI LiDAR odometry and (ii) ModelNet-40 datasets shows that our method outperforms prior state-of-the-art methods - e.g., on the KITTI data set, DCP-v2 by1.3 and 1.5 times, and PointNetLK by 1.8 and 1.9 times better rotational and translational accuracy respectively. Evaluation on ModelNet40 shows that RPSRNet is more robust than other benchmark methods when the samples contain a significant amount of noise and other disturbances. RPSRNet accurately registers point clouds with non-uniform sampling densities, e.g., LiDAR data, which cannot be processed by many existing deep-learning-based registration methods.
Implementing an expert system to evaluate technical solutions innovativeness
Venue: arXiv cs.AI 2021
Year: 2021

The paper presents a possible solution to the problem of algorithmization for quantifying inno-vativeness indicators of technical products, inventions and technologies. The concepts of technological nov-elty, relevance and implementability as components of product innovation criterion are introduced. Authors propose a model and algorithm to calculate every of these indicators of innovativeness under conditions of incompleteness and inaccuracy, and sometimes inconsistency of the initial information. The paper describes the developed specialized software that is a promising methodological tool for using interval estimations in accordance with the theory of evidence. These estimations are used in the analysis of complex multicomponent systems, aggregations of large volumes of fuzzy and incomplete data of various structures. Composition and structure of a multi-agent expert system are presented. The purpose of such system is to process groups of measurement results and to estimate indicators values of objects innovativeness. The paper defines active elements of the system, their functionality, roles, interaction order, input and output inter-faces, as well as the general software functioning algorithm. It describes implementation of software modules and gives an example of solving a specific problem to determine the level of technical products innovation.
Contrastive Learning of Global-Local Video Representations
Venue: arXiv cs.LG 2021
Year: 2021

Contrastive learning has delivered impressive results for various tasks in the self-supervised regime. However, existing approaches optimize for learning representations specific to downstream scenarios, i.e., \textit{global} representations suitable for tasks such as classification or \textit{local} representations for tasks such as detection and localization. While they produce satisfactory results in the intended downstream scenarios, they often fail to generalize to tasks that they were not originally designed for. In this work, we propose to learn video representations that generalize to both the tasks which require global semantic information (e.g., classification) and the tasks that require local fine-grained spatio-temporal information (e.g., localization). We achieve this by optimizing two contrastive objectives that together encourage our model to learn global-local visual information given audio signals. We show that the two objectives mutually improve the generalizability of the learned global-local representations, significantly outperforming their disjointly learned counterparts. We demonstrate our approach on various tasks including action/sound classification, lip reading, deepfake detection, event and sound localization (https://github.com/yunyikristy/global\_local).
Deep Transformer Networks for Time Series Classification: The NPP Safety Case
Venue: arXiv cs.LG 2021
Year: 2021

A challenging part of dynamic probabilistic risk assessment for nuclear power plants is the need for large amounts of temporal simulations given various initiating events and branching conditions from which representative feature extraction becomes complicated for subsequent applications. Artificial Intelligence techniques have been shown to be powerful tools in time-dependent sequential data processing to automatically extract and yield complex features from large data. An advanced temporal neural network referred to as the Transformer is used within a supervised learning fashion to model the time-dependent NPP simulation data and to infer whether a given sequence of events leads to core damage or not. The training and testing datasets for the Transformer are obtained by running 10,000 RELAP5-3D NPP blackout simulations with the list of variables obtained from the RAVEN software. Each simulation is classified as "OK" or "CORE DAMAGE" based on the consequence. The results show that the Transformer can learn the characteristics of the sequential data and yield promising performance with approximately 99% classification accuracy on the testing dataset.
TermAdventure: Interactively Teaching UNIX Command Line, Text Adventure Style
Venue: arXiv cs.CY 2021
Year: 2021

Introductory UNIX courses are typically organized as lectures, accompanied by a set of exercises, whose solutions are submitted to and reviewed by the lecturers. While this arrangement has become standard practice, it often requires the use of an external tool or interface for submission and does not automatically check its correctness. That in turn leads to increased workload and makes it difficult to deal with potential plagiarism. In this work we present TermAdventure (TA), a suite of tools for creating interactive UNIX exercises. These resemble text adventure games, which immerse the user in a text environment and let them interact with it using textual commands. In our case the ''adventure'' takes place inside a UNIX system and the user interaction happens via the standard UNIX command line. The adventure is a set of exercises, which are presented and automatically evaluated by the system, all from within the command line environment. The suite is released under an open source license, has minimal dependencies and can be used either on a UNIX-style server or a desktop computer running any major OS platform through Docker. We also reflect on our experience of using the presented suite as the primary teaching tool for an introductory UNIX course for Data Scientists and discuss the implications of its deployment in similar courses. The suite is released under the terms of an open-source license at \url{https://github.com/NaiveNeuron/TermAdventure}.
DockerMock: Pre-Build Detection of Dockerfile Faults through Mocking Instruction Execution
Venue: arXiv cs.SE 2021
Year: 2021

Continuous Integration (CI) and Continuous Deployment (CD) are widely adopted in software engineering practice. In reality, the CI/CD pipeline execution is not yet reliably continuous because it is often interrupted by Docker build failures. However, the existing trial-and-error practice to detect faults is time-consuming. To timely detect Dockerfile faults, we propose a context-based pre-build analysis approach, named DockerMock, through mocking the execution of common Dockerfile instructions. A Dockerfile fault is declared when an instruction conflicts with the approximated and accumulated running context. By explicitly keeping track of whether the context is fuzzy, DockerMock strikes a good balance of detection precision and recall. We evaluated DockerMock with 53 faults in 41 Dockerfiles from open source projects on GitHub and 130 faults in 105 Dockerfiles from student course projects. On average, DockerMock detected 68.0% Dockerfile faults in these two datasets. While baseline hadolint detected 6.5%, and baseline BuildKit detected 60.5% without instruction execution. In the GitHub dataset, DockerMock reduces the number of builds to 47, outperforming that of hadolint (73) and BuildKit (74).