OpenRaiser/Intern-Atlas huggingface dataset:
Decentralized Langevin Dynamics over a Directed Graph
Venue: arXiv cs.DC 2021
Year: 2021

The prevalence of technologies in the space of the Internet of Things and use of multi-processing computing platforms to aid in the computation required to perform learning and inference from large volumes of data has necessitated the extensive study of algorithms on decentralized platforms. In these settings, computing nodes send and receive data across graph-structured communication links, and using a combination of local computation and consensus-seeking communication, cooperately solve a problem of interest. Recently, Langevin dynamics as a tool for high dimensional sampling and posterior Bayesian inference has been studied in the context of a decentralized operation. However, this work has been limited to undirected graphs, wherein all communication is two-sided, i.e., if node A can send data to node B, then node B can also send data to node A. We extend the state of the art in considering Langevin dynamics on directed graphs.
Extended Tree Search for Robot Task and Motion Planning
Venue: arXiv cs.RO 2021
Year: 2021

Integrated task and motion planning (TAMP) is desirable for generalized autonomy robots but it is challenging at the same time. TAMP requires the planner to not only search in both the large symbolic task space and the high-dimension motion space but also deal with the infeasible task actions due to its intrinsic hierarchical process. We propose a novel decision-making framework for TAMP by constructing an extended decision tree for both symbolic task planning and high-dimension motion variable binding. We integrate top-k planning for generating explicitly a skeleton space where a variety of candidate skeleton plans are at disposal. Moreover, we effectively combine this skeleton space with the resultant motion variable spaces into a single extended decision space. Accordingly, we use Monte-Carlo Tree Search (MCTS) to ensure an exploration-exploitation balance at each decision node and optimize globally to produce optimal solutions. The proposed seamless combination of symbolic top-k planning with streams, with the proved optimality of MCTS, leads to a powerful planning algorithm that can handle the combinatorial complexity of long-horizon manipulation tasks. We empirically evaluate our proposed algorithm in challenging robot tasks with different domains that require multi-stage decisions and show how our method can overcome the large task space and motion space through its effective tree search compared to its most competitive baseline method.
Ontology-based industrial data management platform
Venue: arXiv cs.DB 2021
Year: 2021

Relational and noSQL storages are developed for the fast processing of the large data sets having a stable structure, while the ontologies are used to rep-resent complex and dynamic sets of information of a limited size. In the in-dustrial applications it is often needed to maintain the large warehouses of data consolidated from various sources. The ontologies are useful to repre-sent the structure of that data, but RDF triple stores are not well suitable for storing it. We offer an approach and a system allowing to use the opportuni-ties of fast storage engines along with the flexibility of ontology-based data management tools, including SPARQL queries. The system implements a multi-model data abstraction layer which allows working with the data as if it is situated in RDF triple store, executes SPARQL queries over it and ap-plies SHACL constraints and rules.
Low-level cognitive skill transfer between two individuals' minds via computer game-based framework
Venue: arXiv cs.HC 2021
Year: 2021

The novel technique introduced here aims to accomplish the first stage of transferring low-level cognitive skills between two individuals (e.g. from expert to learner) to ease the consecutive higher level declarative learning process for the target "learner" individual in a game environment. Such low-level cognitive skill is associated with the procedural knowledge and established at low-level of mind which can be unveiled and transferred by only a novel technique (rather than by a traditional educational environment ) like a highly interactive computer game domain in which a user exposes his/her unconscious mind behaviors via the game-hero non-deliberately during the game sessions. The cognitive data exposed by the game-hero would be recorded, and then be modelled by the artificial intelligence technique like Bayesian networks for an early stage of cognitive skill transfer and the cognitive stimuli are also generated to be used as game agents to train the learner.
hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices
Venue: arXiv cs.LG 2021
Year: 2021

Accessible machine learning algorithms, software, and diagnostic tools for energy-efficient devices and systems are extremely valuable across a broad range of application domains. In scientific domains, real-time near-sensor processing can drastically improve experimental design and accelerate scientific discoveries. To support domain scientists, we have developed hls4ml, an open-source software-hardware codesign workflow to interpret and translate machine learning algorithms for implementation with both FPGA and ASIC technologies. We expand on previous hls4ml work by extending capabilities and techniques towards low-power implementations and increased usability: new Python APIs, quantization-aware pruning, end-to-end FPGA workflows, long pipeline kernels for low power, and new device backends include an ASIC workflow. Taken together, these and continued efforts in hls4ml will arm a new generation of domain scientists with accessible, efficient, and powerful tools for machine-learning-accelerated discovery.
SimTriplet: Simple Triplet Representation Learning with a Single GPU
Venue: arXiv cs.CV 2021
Year: 2021

Contrastive learning is a key technique of modern self-supervised learning. The broader accessibility of earlier approaches is hindered by the need of heavy computational resources (e.g., at least 8 GPUs or 32 TPU cores), which accommodate for large-scale negative samples or momentum. The more recent SimSiam approach addresses such key limitations via stop-gradient without momentum encoders. In medical image analysis, multiple instances can be achieved from the same patient or tissue. Inspired by these advances, we propose a simple triplet representation learning (SimTriplet) approach on pathological images. The contribution of the paper is three-fold: (1) The proposed SimTriplet method takes advantage of the multi-view nature of medical images beyond self-augmentation; (2) The method maximizes both intra-sample and inter-sample similarities via triplets from positive pairs, without using negative samples; and (3) The recent mix precision training is employed to advance the training by only using a single GPU with 16GB memory. By learning from 79,000 unlabeled pathological patch images, SimTriplet achieved 10.58% better performance compared with supervised learning. It also achieved 2.13% better performance compared with SimSiam. Our proposed SimTriplet can achieve decent performance using only 1% labeled data. The code and data are available at https://github.com/hrlblab/SimTriple.
One-Sided $k$-Orthogonal Matrices Over Finite Semi-Local Rings And Their Codes
Venue: arXiv cs.IT 2021
Year: 2021

Let $R$ be a finite commutative ring with unity $1_R$ and $k \in R$. Properties of one-sided $k$-orthogonal $n \times n$ matrices over $R$ are presented. When $k$ is idempotent, these matrices form a semigroup structure. Consequently new families of matrix semigroups over certain finite semi-local rings are constructed. When $k=1_R$, the classical orthogonal group of degree $n$ is obtained. It is proved that, if $R$ is a semi-local ring, then these semigroups are isomorphic to a finite product of $k$-orthogonal semigroups over fields. Finally, the antiorthogonal and self-orthogonal matrices that give rise to leading-systematic self-dual or weakly self-dual linear codes are discussed.
Flow Time Scheduling with Uncertain Processing Time
Venue: arXiv cs.DS 2021
Year: 2021

We consider the problem of online scheduling on a single machine in order to minimize weighted flow time. The existing algorithms for this problem (STOC '01, SODA '03, FOCS '18) all require exact knowledge of the processing time of each job. This assumption is crucial, as even a slight perturbation of the processing time would lead to polynomial competitive ratio. However, this assumption very rarely holds in real-life scenarios. In this paper, we present the first algorithm for weighted flow time which do not require exact knowledge of the processing times of jobs. Specifically, we introduce the Scheduling with Predicted Processing Time (SPPT) problem, where the algorithm is given a prediction for the processing time of each job, instead of its real processing time. For the case of a constant factor distortion between the predictions and the real processing time, our algorithms match all the best known competitiveness bounds for weighted flow time -- namely $O(\log P), O(\log D)$ and $O(\log W)$, where $P,D,W$ are the maximum ratios of processing times, densities, and weights, respectively. For larger errors, the competitiveness of our algorithms degrades gracefully.
Challenges for Reinforcement Learning in Healthcare
Venue: arXiv cs.LG 2021
Year: 2021

Many healthcare decisions involve navigating through a multitude of treatment options in a sequential and iterative manner to find an optimal treatment pathway with the goal of an optimal patient outcome. Such optimization problems may be amenable to reinforcement learning. A reinforcement learning agent could be trained to provide treatment recommendations for physicians, acting as a decision support tool. However, a number of difficulties arise when using RL beyond benchmark environments, such as specifying the reward function, choosing an appropriate state representation and evaluating the learned policy.
An Amharic News Text classification Dataset
Venue: arXiv cs.CL 2021
Year: 2021

In NLP, text classification is one of the primary problems we try to solve and its uses in language analyses are indisputable. The lack of labeled training data made it harder to do these tasks in low resource languages like Amharic. The task of collecting, labeling, annotating, and making valuable this kind of data will encourage junior researchers, schools, and machine learning practitioners to implement existing classification models in their language. In this short paper, we aim to introduce the Amharic text classification dataset that consists of more than 50k news articles that were categorized into 6 classes. This dataset is made available with easy baseline performances to encourage studies and better performance experiments.
Exploiting Asynchronous Priority Scheduling in Parallel Eikonal Solvers
Venue: arXiv cs.DC 2021
Year: 2021

Numerical solutions to the Eikonal equation are computed using variants of the fast marching method, the fast sweeping method, and the fast iterative method. In this paper, we provide a unified view of these algorithms that highlights their similarities and suggests a wider class of Eikonal solvers. We then use this framework to justify applying concurrent priority scheduling techniques to Eikonal solvers. We demonstrate that doing so results in good parallel performance for a problem from seismology. We explain why existing Eikonal solvers may produce different results despite using the same differencing scheme and demonstrate techniques to address these discrepancies. These techniques allow us to obtain deterministic output from our asynchronous fine-grained parallel Eikonal solver.
Structural Connectome Atlas Construction in the Space of Riemannian Metrics
Venue: arXiv cs.CV 2021
Year: 2021

The structural connectome is often represented by fiber bundles generated from various types of tractography. We propose a method of analyzing connectomes by representing them as a Riemannian metric, thereby viewing them as points in an infinite-dimensional manifold. After equipping this space with a natural metric structure, the Ebin metric, we apply object-oriented statistical analysis to define an atlas as the Fréchet mean of a population of Riemannian metrics. We demonstrate connectome registration and atlas formation using connectomes derived from diffusion tensors estimated from a subset of subjects from the Human Connectome Project.
Regret Bounds for Generalized Linear Bandits under Parameter Drift
Venue: arXiv cs.LG 2021
Year: 2021

Generalized Linear Bandits (GLBs) are powerful extensions to the Linear Bandit (LB) setting, broadening the benefits of reward parametrization beyond linearity. In this paper we study GLBs in non-stationary environments, characterized by a general metric of non-stationarity known as the variation-budget or \emph{parameter-drift}, denoted $B_T$. While previous attempts have been made to extend LB algorithms to this setting, they overlook a salient feature of GLBs which flaws their results. In this work, we introduce a new algorithm that addresses this difficulty. We prove that under a geometric assumption on the action set, our approach enjoys a $\tilde{\mathcal{O}}(B_T^{1/3}T^{2/3})$ regret bound. In the general case, we show that it suffers at most a $\tilde{\mathcal{O}}(B_T^{1/5}T^{4/5})$ regret. At the core of our contribution is a generalization of the projection step introduced in Filippi et al. (2010), adapted to the non-stationary nature of the problem. Our analysis sheds light on central mechanisms inherited from the setting by explicitly splitting the treatment of the learning and tracking aspects of the problem.
Containing Malicious Package Updates in npm with a Lightweight Permission System
Venue: arXiv cs.CR 2021
Year: 2021

The large amount of third-party packages available in fast-moving software ecosystems, such as Node.js/npm, enables attackers to compromise applications by pushing malicious updates to their package dependencies. Studying the npm repository, we observed that many packages in the npm repository that are used in Node.js applications perform only simple computations and do not need access to filesystem or network APIs. This offers the opportunity to enforce least-privilege design per package, protecting applications and package dependencies from malicious updates. We propose a lightweight permission system that protects Node.js applications by enforcing package permissions at runtime. We discuss the design space of solutions and show that our system makes a large number of packages much harder to be exploited, almost for free.
On Modeling Human Perceptions of Allocation Policies with Uncertain Outcomes
Venue: arXiv cs.CY 2021
Year: 2021

Many policies allocate harms or benefits that are uncertain in nature: they produce distributions over the population in which individuals have different probabilities of incurring harm or benefit. Comparing different policies thus involves a comparison of their corresponding probability distributions, and we observe that in many instances the policies selected in practice are hard to explain by preferences based only on the expected value of the total harm or benefit they produce. In cases where the expected value analysis is not a sufficient explanatory framework, what would be a reasonable model for societal preferences over these distributions? Here we investigate explanations based on the framework of probability weighting from the behavioral sciences, which over several decades has identified systematic biases in how people perceive probabilities. We show that probability weighting can be used to make predictions about preferences over probabilistic distributions of harm and benefit that function quite differently from expected-value analysis, and in a number of cases provide potential explanations for policy preferences that appear hard to motivate by other means. In particular, we identify optimal policies for minimizing perceived total harm and maximizing perceived total benefit that take the distorting effects of probability weighting into account, and we discuss a number of real-world policies that resemble such allocational strategies. Our analysis does not provide specific recommendations for policy choices, but is instead fundamentally interpretive in nature, seeking to describe observed phenomena in policy choices.
Interpretable bias mitigation for textual data: Reducing gender bias in patient notes while maintaining classification performance
Venue: arXiv cs.CL 2021
Year: 2021

Medical systems in general, and patient treatment decisions and outcomes in particular, are affected by bias based on gender and other demographic elements. As language models are increasingly applied to medicine, there is a growing interest in building algorithmic fairness into processes impacting patient care. Much of the work addressing this question has focused on biases encoded in language models -- statistical estimates of the relationships between concepts derived from distant reading of corpora. Building on this work, we investigate how word choices made by healthcare practitioners and language models interact with regards to bias. We identify and remove gendered language from two clinical-note datasets and describe a new debiasing procedure using BERT-based gender classifiers. We show minimal degradation in health condition classification tasks for low- to medium-levels of bias removal via data augmentation. Finally, we compare the bias semantically encoded in the language models with the bias empirically observed in health records. This work outlines an interpretable approach for using data augmentation to identify and reduce the potential for bias in natural language processing pipelines.
A Distributed, Decoupled System for Losslessly Streaming Dynamic Light Probes to Thin Clients
Venue: arXiv cs.DC 2021
Year: 2021

We present a networked, high performance graphics system that combines dynamic, high quality, ray traced global illumination computed on a server with direct illumination and primary visibility computed on a client. This approach provides many of the image quality benefits of real-time ray tracing on low-power and legacy hardware, while maintaining a low latency response and mobile form factor. Our system distributes the graphic pipeline over a network by computing diffuse global illumination on a remote machine. Global illumination is computed using a recent irradiance volume representation combined with a novel, lossless, HEVC-based, hardware-accelerated encoding, and a perceptually-motivated update scheme. Our experimental implementation streams thousands of irradiance probes per second and requires less than 50 Mbps of throughput, reducing the consumed bandwidth by 99.4% when streaming at 60 Hz compared to traditional lossless texture compression. This bandwidth reduction allows higher quality and lower latency graphics than state-of-the-art remote rendering via video streaming. In addition, our split-rendering solution decouples remote computation from local rendering and so does not limit local display update rate or resolution.
DeepCPCFG: Deep Learning and Context Free Grammars for End-to-End Information Extraction
Venue: arXiv cs.CL 2021
Year: 2021

We address the challenge of extracting structured information from business documents without detailed annotations. We propose Deep Conditional Probabilistic Context Free Grammars (DeepCPCFG) to parse two-dimensional complex documents and use Recursive Neural Networks to create an end-to-end system for finding the most probable parse that represents the structured information to be extracted. This system is trained end-to-end with scanned documents as input and only relational-records as labels. The relational-records are extracted from existing databases avoiding the cost of annotating documents by hand. We apply this approach to extract information from scanned invoices achieving state-of-the-art results despite using no hand-annotations.
Self-Learning for Zero Shot Neural Machine Translation
Venue: arXiv cs.CL 2021
Year: 2021

Neural Machine Translation (NMT) approaches employing monolingual data are showing steady improvements in resource rich conditions. However, evaluations using real-world low-resource languages still result in unsatisfactory performance. This work proposes a novel zero-shot NMT modeling approach that learns without the now-standard assumption of a pivot language sharing parallel data with the zero-shot source and target languages. Our approach is based on three stages: initialization from any pre-trained NMT model observing at least the target language, augmentation of source sides leveraging target monolingual data, and learning to optimize the initial model to the zero-shot pair, where the latter two constitute a self-learning cycle. Empirical findings involving four diverse (in terms of a language family, script and relatedness) zero-shot pairs show the effectiveness of our approach with up to +5.93 BLEU improvement against a supervised bilingual baseline. Compared to unsupervised NMT, consistent improvements are observed even in a domain-mismatch setting, attesting to the usability of our method.
Beyond Self-Supervision: A Simple Yet Effective Network Distillation Alternative to Improve Backbones
Venue: arXiv cs.CV 2021
Year: 2021

Recently, research efforts have been concentrated on revealing how pre-trained model makes a difference in neural network performance. Self-supervision and semi-supervised learning technologies have been extensively explored by the community and are proven to be of great potential in obtaining a powerful pre-trained model. However, these models require huge training costs (i.e., hundreds of millions of images or training iterations). In this paper, we propose to improve existing baseline networks via knowledge distillation from off-the-shelf pre-trained big powerful models. Different from existing knowledge distillation frameworks which require student model to be consistent with both soft-label generated by teacher model and hard-label annotated by humans, our solution performs distillation by only driving prediction of the student model consistent with that of the teacher model. Therefore, our distillation setting can get rid of manually labeled data and can be trained with extra unlabeled data to fully exploit capability of teacher model for better learning. We empirically find that such simple distillation settings perform extremely effective, for example, the top-1 accuracy on ImageNet-1k validation set of MobileNetV3-large and ResNet50-D can be significantly improved from 75.2% to 79% and 79.1% to 83%, respectively. We have also thoroughly analyzed what are dominant factors that affect the distillation performance and how they make a difference. Extensive downstream computer vision tasks, including transfer learning, object detection and semantic segmentation, can significantly benefit from the distilled pretrained models. All our experiments are implemented based on PaddlePaddle, codes and a series of improved pretrained models with ssld suffix are available in PaddleClas.