OpenRaiser/Intern-Atlas huggingface dataset:
Trainable Spectrally Initializable Matrix Transformations in Convolutional Neural Networks
Venue: arXiv cs.CV 2019
Year: 2019

In this work, we investigate the application of trainable and spectrally initializable matrix transformations on the feature maps produced by convolution operations. While previous literature has already demonstrated the possibility of adding static spectral transformations as feature processors, our focus is on more general trainable transforms. We study the transforms in various architectural configurations on four datasets of different nature: from medical (ColorectalHist, HAM10000) and natural (Flowers, ImageNet) images to historical documents (CB55) and handwriting recognition (GPDS). With rigorous experiments that control for the number of parameters and randomness, we show that networks utilizing the introduced matrix transformations outperform vanilla neural networks. The observed accuracy increases by an average of 2.2 across all datasets. In addition, we show that the benefit of spectral initialization leads to significantly faster convergence, as opposed to randomly initialized matrix transformations. The transformations are implemented as auto-differentiable PyTorch modules that can be incorporated into any neural network architecture. The entire code base is open-source.
Detecting Patterns of Physiological Response to Hemodynamic Stress via Unsupervised Deep Learning
Venue: arXiv cs.LG 2019
Year: 2019

Monitoring physiological responses to hemodynamic stress can help in determining appropriate treatment and ensuring good patient outcomes. Physicians' intuition suggests that the human body has a number of physiological response patterns to hemorrhage which escalate as blood loss continues, however the exact etiology and phenotypes of such responses are not well known or understood only at a coarse level. Although previous research has shown that machine learning models can perform well in hemorrhage detection and survival prediction, it is unclear whether machine learning could help to identify and characterize the underlying physiological responses in raw vital sign data. We approach this problem by first transforming the high-dimensional vital sign time series into a tractable, lower-dimensional latent space using a dilated, causal convolutional encoder model trained purely unsupervised. Second, we identify informative clusters in the embeddings. By analyzing the clusters of latent embeddings and visualizing them over time, we hypothesize that the clusters correspond to the physiological response patterns that match physicians' intuition. Furthermore, we attempt to evaluate the latent embeddings using a variety of methods, such as predicting the cluster labels using explainable features.
Incentive Compatible Active Learning
Venue: arXiv cs.GT 2019
Year: 2019

We consider active learning under incentive compatibility constraints. The main application of our results is to economic experiments, in which a learner seeks to infer the parameters of a subject's preferences: for example their attitudes towards risk, or their beliefs over uncertain events. By cleverly adapting the experimental design, one can save on the time spent by subjects in the laboratory, or maximize the information obtained from each subject in a given laboratory session; but the resulting adaptive design raises complications due to incentive compatibility. A subject in the lab may answer questions strategically, and not truthfully, so as to steer subsequent questions in a profitable direction. We analyze two standard economic problems: inference of preferences over risk from multiple price lists, and belief elicitation in experiments on choice over uncertainty. In the first setting, we tune a simple and fast learning algorithm to retain certain incentive compatibility properties. In the second setting, we provide an incentive compatible learning algorithm based on scoring rules with query complexity that differs from obvious methods of achieving fast learning rates only by subpolynomial factors. Thus, for these areas of application, incentive compatibility may be achieved without paying a large sample complexity price.
92c/MFlops/s, Ultra-Large-Scale Neural-Network Training on a PIII Cluster
Venue: arXiv cs.LG 2019
Year: 2019

Artificial neural networks with millions of adjustable parameters and a similar number of training examples are a potential solution for difficult, large-scale pattern recognition problems in areas such as speech and face recognition, classification of large volumes of web data, and finance. The bottleneck is that neural network training involves iterative gradient descent and is extremely computationally intensive. In this paper we present a technique for distributed training of Ultra Large Scale Neural Networks (ULSNN) on Bunyip, a Linux-based cluster of 196 Pentium III processors. To illustrate ULSNN training we describe an experiment in which a neural network with 1.73 million adjustable parameters was trained to recognize machine-printed Japanese characters from a database containing 9 million training patterns. The training runs with a average performance of 163.3 GFlops/s (single precision). With a machine cost of \$150,913, this yields a price/performance ratio of 92.4c/MFlops/s (single precision). For comparison purposes, training using double precision and the ATLAS DGEMM produces a sustained performance of 70 MFlops/s or \$2.16 / MFlop/s (double precision).
Pose estimation and bin picking for deformable products
Venue: arXiv cs.RO 2019
Year: 2019

Robotic systems in manufacturing applications commonly assume known object geometry and appearance. This simplifies the task for the 3D perception algorithms and allows the manipulation to be more deterministic. However, those approaches are not easily transferable to the agricultural and food domains due to the variability and deformability of natural food. We demonstrate an approach applied to poultry products that allows picking up a whole chicken from an unordered bin using a suction cup gripper, estimating its pose using a Deep Learning approach, and placing it in a canonical orientation where it can be further processed. Our robotic system was experimentally evaluated and is able to generalize to object variations and achieves high accuracy on bin picking and pose estimation tasks in a real-world environment.
Locking-free Simulation of Isometric Thin Plates
Venue: arXiv cs.GR 2019
Year: 2019

To efficiently simulate very thin, inextensible materials like cloth or paper, it is tempting to replace force-based thin-plate dynamics with hard isometry constraints. Unfortunately, naive formulations of the constraints induce membrane locking---artificial stiffening of bending modes due to the inability of discrete kinematics to reproduce exact isometries. We propose a simple set of meshless isometry constraints, based on moving-least-squares averaging of the strain tensor, which do not lock, and which can be easily incorporated into standard constrained Lagrangian dynamics integration.
Oblivious Permutations on the Plane
Venue: arXiv cs.DC 2019
Year: 2019

We consider a distributed system of n identical mobile robots operating in the two dimensional Euclidian plane. As in the previous studies, we consider the robots to be anonymous, oblivious, dis-oriented, and without any communication capabilities, operating based on the Look-Compute-Move model where the next location of a robot depends only on its view of the current configuration. Even in this seemingly weak model, most formation problems which require constructing specific configurations, can be solved quite easily when the robots are fully synchronized with each other. In this paper we introduce and study a new class of problems which, unlike the formation problems so far, cannot always be solved even in the fully synchronous model with atomic and rigid moves. This class of problems requires the robots to permute their locations in the plane. In particular, we are interested in implementing two special types of permutations -- permutations without any fixed points and permutations of order $n$. The former (called MOVE-ALL) requires each robot to visit at least two of the initial locations, while the latter (called VISIT-ALL) requires every robot to visit each of the initial locations in a periodic manner. We provide a characterization of the solvability of these problems, showing the main challenges in solving this class of problems for mobile robots. We also provide algorithms for the feasible cases, in particular distinguishing between one-step algorithms (where each configuration must be a permutation of the original configuration) and multi-step algorithms (which allow intermediate configurations). These results open a new research direction in mobile distributed robotics which has not been investigated before.
The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design
Venue: arXiv cs.LG 2019
Year: 2019

The past decade has seen a remarkable series of advances in machine learning, and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas, including computer vision, speech recognition, language translation, and natural language understanding tasks. This paper is a companion paper to a keynote talk at the 2020 International Solid-State Circuits Conference (ISSCC) discussing some of the advances in machine learning, and their implications on the kinds of computational devices we need to build, especially in the post-Moore's Law-era. It also discusses some of the ways that machine learning may also be able to help with some aspects of the circuit design process. Finally, it provides a sketch of at least one interesting direction towards much larger-scale multi-task models that are sparsely activated and employ much more dynamic, example- and task-based routing than the machine learning models of today.
Radio Resource Allocation in 5G New Radio: A Neural Networks Based Approach)
Venue: arXiv cs.NI 2019
Year: 2019

The minimum frequency-time unit that can be allocated to User Equipments (UEs) in the fifth generation (5G) cellular networks is a Resource Block (RB). A RB is a channel composed of a set of OFDM subcarriers for a given time slot duration. 5G New Radio (NR) allows for a large number of block shapes ranging from 15 kHz to 480 kHz. In this paper, we address the problem of RBs allocation to UEs. The RBs are allocated at the beginning of each time slot based on the channel state of each UE. The problem is formulated based on the Generalized Proportional Fair (GPF) scheduling. Then, we model the problem as a 2-Dimension Hopfield Neural Networks (2D-HNN). Finally, in an attempt to solve the problem, the energy function of 2D-HNN is investigated. Simulation results show the efficiency of the proposed approach.
Improving the Space-Time Efficiency of Processor-Oblivious Matrix Multiplication Algorithms
Venue: arXiv cs.DC 2019
Year: 2019

Classic cache-oblivious parallel matrix multiplication algorithms achieve optimality either in time or space, but not both, which promotes lots of research on the best possible balance or tradeoff of such algorithms. We study modern processor-oblivious runtime systems and figure out several ways to improve algorithm's time bound while still bounding space and cache requirements to be asymptotically optimal. By our study, we give out sublinear time, optimal work, space and cache algorithms for both general matrix multiplication on a semiring and Strassen-like fast algorithm. Our experiments also show such algorithms have empirical advantages over classic counterparts. Our study provides new insights and research angles on how to optimize cache-oblivious parallel algorithms from both theoretical and empirical perspectives.
Nested Dataflow Algorithms for Dynamic Programming Recurrences with more than O(1) Dependency
Venue: arXiv cs.DS 2019
Year: 2019

Dynamic programming problems have wide applications in real world and have been studied extensively in both serial and parallel settings. In 1994, Galil and Park developed work-efficient and sublinear-time algorithms for several important dynamic programming problems based on the closure method and matrix product method. However, in the same paper, they raised an open question whether such an algorithm exists for the general GAP problem. % In this paper, we answer their question by developing the first work-efficient and sublinear-time GAP algorithm based on the closure method and Nested Dataflow method. % We also improve the time bounds of classic work-efficient, cache-oblivious and cache-efficient algorithms for the 1D problem and GAP problem, respectively.
Development of a Secure and Private Electronic Procurement System based on Blockchain Implementation
Venue: arXiv cs.CR 2019
Year: 2019

This paper presents the development of an online procurement system and the integration of blockchain technology. Various tools such as PHP, JavaScript, HTML, CSS, and jQuery were used in designing the graphical, programming logic, and blockchain aspect of the system. Every page and function will have their respective construction and result. In addition, the proposed system's flow of process and the methods on the testing and hosting of the site as well as the different web development languages used in every part of the development and design process were presented. The proposed system was successfully and functionally developed starting from the execution of procurement proper, to the placement of procured items or goods, and up to the signing of contracts by the winner and the procurer. Lastly, features were added such as user profiles of the bidder and procurer.
Reinforcement Learning-Driven Test Generation for Android GUI Applications using Formal Specifications
Venue: arXiv cs.SE 2019
Year: 2019

There have been many studies on automated test generation for mobile Graphical User Interface (GUI) applications. These studies successfully demonstrate how to detect fatal exceptions and achieve high code and activity coverage with fully automated test generation engines. However, it is unclear how many GUI functions these engines manage to test. Furthermore, these engines implement only implicit test oracles. We propose Fully Automated Reinforcement LEArning-Driven Specification-Based Test Generator for Android (FARLEAD-Android). FARLEAD-Android accepts a GUI-level formal specification as a Linear-time Temporal Logic (LTL) formula. By dynamically executing the Application Under Test (AUT), it learns how to generate a test that satisfies the LTL formula using Reinforcement Learning (RL). The LTL formula does not just guide the test generation but also acts as a specified test oracle, enabling the developer to define automated test oracles for a wide variety of GUI functions by changing the formula. Our evaluation shows that FARLEAD-Android is more effective and achieves higher performance in generating tests for specified GUI functions than three known approaches, Random, Monkey, and QBEa. To the best of our knowledge, FARLEAD-Android is the first fully automated mobile GUI testing engine that uses formal specifications.
Identification of Rhetorical Roles of Sentences in Indian Legal Judgments
Venue: arXiv cs.IR 2019
Year: 2019

Automatically understanding the rhetorical roles of sentences in a legal case judgement is an important problem to solve, since it can help in several downstream tasks like summarization of legal judgments, legal search, and so on. The task is challenging since legal case documents are usually not well-structured, and these rhetorical roles may be subjective (as evident from variation of opinions between legal experts). In this paper, we address this task for judgments from the Supreme Court of India. We label sentences in 50 documents using multiple human annotators, and perform an extensive analysis of the human-assigned labels. We also attempt automatic identification of the rhetorical roles of sentences. While prior approaches towards this task used Conditional Random Fields over manually handcrafted features, we explore the use of deep neural models which do not require hand-crafting of features. Experiments show that neural models perform much better in this task than baseline methods which use handcrafted features.
Attentive Geo-Social Group Recommendation
Venue: arXiv cs.SI 2019
Year: 2019

Social activities play an important role in people's daily life since they interact. For recommendations based on social activities, it is vital to have not only the activity information but also individuals' social relations. Thanks to the geo-social networks and widespread use of location-aware mobile devices, massive geo-social data is now readily available for exploitation by the recommendation system. In this paper, a novel group recommendation method, called attentive geo-social group recommendation, is proposed to recommend the target user with both activity locations and a group of users that may join the activities. We present an attention mechanism to model the influence of the target user $u_T$ in candidate user groups that satisfy the social constraints. It helps to retrieve the optimal user group and activity topic candidates, as well as explains the group decision-making process. Once the user group and topics are retrieved, a novel efficient spatial query algorithm SPA-DF is employed to determine the activity location under the constraints of the given user group and activity topic candidates. The proposed method is evaluated in real-world datasets and the experimental results show that the proposed model significantly outperforms baseline methods.
The Potential Gains of Macrodiversity in mmWave Cellular Networks with Correlated Blocking
Venue: arXiv cs.IT 2019
Year: 2019

At millimeter wave (mmWave) frequencies, signals are prone to blocking by objects in the environment, which causes paths to go from line-of-sight (LOS) to non-LOS (NLOS). We consider macrodiversity as a strategy to improve the performance of mmWave cellular systems, where the user attempts to connect with two or more base stations. An accurate analysis of macrodiversity must account for the possibility of correlated blocking, which occurs when a single blockage simultaneously blocks the paths to two base stations. In this paper, we analyze the macrodiverity gain in the presence of correlated random blocking and interference. To do so, we develop a framework to determine distributions for the LOS probability, SNR, and SINR by taking into account correlated blocking. We consider a cellular uplink with both diversity combining and selection combining schemes. We also study the impact of blockage size and blockage density. We show that blocking can be both a blessing and a curse. On the one hand, the signal from the source transmitter could be blocked, and on the other hand, interfering signals tend to also be blocked, which leads to a completely different effect on macrodiversity gains. We also show that the assumption of independent blocking can lead to an incorrect evaluation of macrodiversity gain, as the correlation tends to decrease macrodiversity gain.
Avoiding hashing and encouraging visual semantics in referential emergent language games
Venue: arXiv cs.LG 2019
Year: 2019

There has been an increasing interest in the area of emergent communication between agents which learn to play referential signalling games with realistic images. In this work, we consider the signalling game setting of Havrylov and Titov and investigate the effect of the feature extractor's weights and of the task being solved on the visual semantics learned or captured by the models. We impose various augmentation to the input images and additional tasks in the game with the aim to induce visual representations which capture conceptual properties of images. Through our set of experiments, we demonstrate that communication systems which capture visual semantics can be learned in a completely self-supervised manner by playing the right types of game.
Adapting and evaluating a deep learning language model for clinical why-question answering
Venue: arXiv cs.CL 2019
Year: 2019

Objectives: To adapt and evaluate a deep learning language model for answering why-questions based on patient-specific clinical text. Materials and Methods: Bidirectional encoder representations from transformers (BERT) models were trained with varying data sources to perform SQuAD 2.0 style why-question answering (why-QA) on clinical notes. The evaluation focused on: 1) comparing the merits from different training data, 2) error analysis. Results: The best model achieved an accuracy of 0.707 (or 0.760 by partial match). Training toward customization for the clinical language helped increase 6% in accuracy. Discussion: The error analysis suggested that the model did not really perform deep reasoning and that clinical why-QA might warrant more sophisticated solutions. Conclusion: The BERT model achieved moderate accuracy in clinical why-QA and should benefit from the rapidly evolving technology. Despite the identified limitations, it could serve as a competent proxy for question-driven clinical information extraction.
Traffic Sign Detection and Recognition for Autonomous Driving in Virtual Simulation Environment
Venue: arXiv cs.CV 2019
Year: 2019

This study developed a traffic sign detection and recognition algorithm based on the RetinaNet. Two main aspects were revised to improve the detection of traffic signs: image cropping to address the issue of large image and small traffic signs; and using more anchors with various scales to detect traffic signs with different sizes and shapes. The proposed algorithm was trained and tested in a series of autonomous driving front-view images in a virtual simulation environment. Results show that the algorithm performed extremely well under good illumination and weather conditions. Its drawbacks are that it sometimes failed to detect object under bad weather conditions like snow and failed to distinguish speed limits signs with different limit values.
Prediction of gaze direction using Convolutional Neural Networks for Autism diagnosis
Venue: arXiv cs.CV 2019
Year: 2019

Autism is a developmental disorder that affects social interaction and communication of children. The gold standard diagnostic tools are very difficult to use and time consuming. However, diagnostic could be deduced from child gaze preferences by looking a video with social and abstract scenes. In this work, we propose an algorithm based on convolutional neural networks to predict gaze direction for a fast and effective autism diagnosis. Early results show that our algorithm achieves real-time response and robust high accuracy for prediction of gaze direction.
Prevalence of code mixing in semi-formal patient communication in low resource languages of South Africa
Venue: arXiv cs.CL 2019
Year: 2019

In this paper we address the problem of code-mixing in resource-poor language settings. We examine data consisting of 182k unique questions generated by users of the MomConnect helpdesk, part of a national scale public health platform in South Africa. We show evidence of code-switching at the level of approximately 10% within this dataset -- a level that is likely to pose challenges for future services. We use a natural language processing library (Polyglot) that supports detection of 196 languages and attempt to evaluate its performance at identifying English, isiZulu and code-mixed questions.