OpenRaiser/Intern-Atlas huggingface dataset:
Seven Myths in Machine Learning Research
Venue: arXiv cs.LG 2019
Year: 2019

We present seven myths commonly believed to be true in machine learning research, circa Feb 2019. This is an archival copy of the blog post at https://crazyoscarchang.github.io/2019/02/16/seven-myths-in-machine-learning-research/ Myth 1: TensorFlow is a Tensor manipulation library Myth 2: Image datasets are representative of real images found in the wild Myth 3: Machine Learning researchers do not use the test set for validation Myth 4: Every datapoint is used in training a neural network Myth 5: We need (batch) normalization to train very deep residual networks Myth 6: Attention $>$ Convolution Myth 7: Saliency maps are robust ways to interpret neural networks
Optimizing Network Performance for Distributed DNN Training on GPU Clusters: ImageNet/AlexNet Training in 1.5 Minutes
Venue: arXiv cs.DC 2019
Year: 2019

It is important to scale out deep neural network (DNN) training for reducing model training time. The high communication overhead is one of the major performance bottlenecks for distributed DNN training across multiple GPUs. Our investigations have shown that popular open-source DNN systems could only achieve 2.5 speedup ratio on 64 GPUs connected by 56 Gbps network. To address this problem, we propose a communication backend named GradientFlow for distributed DNN training, and employ a set of network optimization techniques. First, we integrate ring-based allreduce, mixed-precision training, and computation/communication overlap into GradientFlow. Second, we propose lazy allreduce to improve network throughput by fusing multiple communication operations into a single one, and design coarse-grained sparse communication to reduce network traffic by only transmitting important gradient chunks. When training ImageNet/AlexNet on 512 GPUs, our approach achieves 410.2 speedup ratio and completes 95-epoch training in 1.5 minutes, which outperforms existing approaches.
Using Crowdsourcing to Identify a Proxy of Socio-Economic status
Venue: arXiv cs.CY 2019
Year: 2019

Social Media provides researchers with an unprecedented opportunity to gain insight into various facets of human life. Health practitioners put a great emphasis on pinpointing socioeconomic status (SES) of individuals as they can use to it to predict certain diseases. Crowdsourcing is a term coined that entails gathering intelligence from a user community online. In order to group the users online into communities, researchers have made use of hashtags that will cull the interest of a community of users. In this paper, we propose a mechanism to group a certain group of users based on their geographic background and build a corpus for such users. Specifically, we have looked at discussion forums for some vehi-cles where the site has established communities for different areas to air their grievances or sing the praises of the vehicle. From such a discussion, it was pos-sible to glean the vocabulary that these group of users adheres to. We compared the corpus of different communities and noted the difference in the choice of language. This provided us with the groundwork for predicting the socio-eco-nomic status of such communities that can be particularly helpful to health prac-titioners and in turn used in smart cities to provide better services to the commu-nity members. More work is underway to take words and emojis out of vo-cablary(OOV) and assessing the average score as special cases.
A Quasi-random Algorithm for Anonymous Rendezvous in Heterogeneous Cognitive Radio Networks
Venue: arXiv cs.NI 2019
Year: 2019

The multichannel rendezvous problem that asks two secondary users to rendezvous on a common available channel in a cognitive radio network (CRN) has received a lot of attention lately. Most rendezvous algorithms in the literature focused on constructing channel hopping (CH) sequences that guarantee finite maximum time-to-rendezvous (MTTR). However, these algorithms perform rather poorly in terms of the expected time-to-rendezvous (ETTR) even when compared to the simple random algorithm. In this paper, we propose the quasi-random (QR) CH algorithm that has a comparable ETTR to the random algorithm and a comparable MTTR to the best bound in the literature. Our QR algorithm does not require the unique identifier (ID) assumption and it is very simple to implement in the symmetric, asynchronous, and heterogeneous setting with multiple radios. In a CRN with $N$ commonly labelled channels, the MTTR of the QR algorithm is bounded above by $9 M \lceil n_1/m_1 \rceil \cdot \lceil n_2/m_2 \rceil$ time slots, where $n_1$ (resp. $n_2$) is the number of available channels to user $1$ (resp. 2), $m_1$ (resp. $m_2$) is the number of radios for user $1$ (resp. 2), and $M=\lceil \lceil \log_2 N \rceil /4 \rceil *5+6$. Such a bound is only slightly larger than the best $O((\log \log N) \frac{n_1 n_2}{m_1 m_2})$ bound in the literature. When each SU has a single radio, the ETTR is bounded above by $\frac{n_1 n_2}{G}+9Mn_1n_2 \cdot (1-\frac{G}{n_1 n_2})^M$, where $G$ is the number of common channels between these two users. By conducting extensive simulations, we show that for both the MTTR and the ETTR, our algorithm is comparable to the simple random algorithm and it outperforms several existing algorithms in the literature.
Evaluating model calibration in classification
Venue: arXiv cs.LG 2019
Year: 2019

Probabilistic classifiers output a probability distribution on target classes rather than just a class prediction. Besides providing a clear separation of prediction and decision making, the main advantage of probabilistic models is their ability to represent uncertainty about predictions. In safety-critical applications, it is pivotal for a model to possess an adequate sense of uncertainty, which for probabilistic classifiers translates into outputting probability distributions that are consistent with the empirical frequencies observed from realized outcomes. A classifier with such a property is called calibrated. In this work, we develop a general theoretical calibration evaluation framework grounded in probability theory, and point out subtleties present in model calibration evaluation that lead to refined interpretations of existing evaluation techniques. Lastly, we propose new ways to quantify and visualize miscalibration in probabilistic classification, including novel multidimensional reliability diagrams.
Detecting Heavy Hitters in the Data-plane
Venue: arXiv cs.NI 2019
Year: 2019

The ability to detect, in real-time, heavy hitters is beneficial to many network applications, such as DoS and anomaly detection. Through programmable languages as P4, heavy hitter detection can be implemented directly in the data-plane, allowing custom actions to be applied to packets as they are processed at a network node. This enables networks to immediately respond to changes in network traffic in the data-plane itself and allows for different QoS profiles for heavy hitter and non-heavy hitter traffic. Current interval-based methods that flush the whole counting structure are not well-suited for programmable hardware (the data-plane), because they either require more resources than available in that hardware, they do not provide good accuracy, or require too many actions from the control-plane. A sliding window approach that maintains accuracy over time would solve these issues. However, to the best of our knowledge, the concept of sliding windows in programmable hardware has not been studied yet. In this paper, we develop streaming approaches to detect heavy hitters in the data-plane. We consider the problems of (1) adopting a sliding window and (2) identifying heavy hitters separately and propose multiple memory- and processing-efficient solutions for each of them. These solutions are suitable for P4 programmable hardware and can be combined at will to solve the streaming variant of the heavy hitter detection problem.
Agent Madoff: A Heuristic-Based Negotiation Agent For The Diplomacy Strategy Game
Venue: arXiv cs.MA 2019
Year: 2019

In this paper, we present the strategy of Agent Madoff, which is a heuristic-based negotiation agent that won 2nd place at the Automated Negotiating Agents Competition (ANAC 2017). Agent Madoff is implemented to play the game Diplomacy, which is a strategic board game that mimics the situation during World War I. Each player represents a major European power which has to negotiate with other forces and win possession of a majority supply centers on the map. We propose a design architecture which consists of 3 components: heuristic module, acceptance strategy and bidding strategy. The heuristic module, responsible for evaluating which regions on the graph are more worthy, considers the type of region and the number of supply centers adjacent to the region and return a utility value for each region on the map. The acceptance strategy is done on a case-by-case basis according to the type of the order by calculating the acceptance probability using a composite function. The bidding strategy adopts a defensive approach aimed to neutralize attacks and resolve conflict moves with other players to minimize our loss on supply centers.
Interactive Restriction of a Mobile Robot's Workspace in a Smart Home Environment
Venue: arXiv cs.RO 2019
Year: 2019

Virtual borders are employed to allow humans the interactive and flexible restriction of their mobile robots' workspaces in human-centered environments, e.g. to exclude privacy zones from the workspace or to indicate certain areas for working. They have been successfully specified in interaction processes using methods from human-robot interaction. However, these methods often lack an expressive feedback system, are restricted to robot's on-board interaction capabilities and require a direct line of sight between human and robot. This negatively affects the user experience and interaction time. Therefore, we investigate the effect of a smart environment on the teaching of virtual borders with the objective to enhance the perceptual and interaction capabilities of a robot. For this purpose, we propose a novel interaction method based on a laser pointer, that leverages a smart home environment in the interaction process. This interaction method comprises an architecture for a smart home environment designed to support the interaction process, the cooperation of human, robot and smart environment in the interaction process, a cooperative perception including stationary and mobile cameras to perceive laser spots and an algorithm to extract virtual borders from multiple camera observations. The results of an experimental evaluation support our hypotheses that our novel interaction method features a significantly shorter interaction time and a better user experience compared to an approach without support of a smart environment. Moreover, the interaction method does not negatively affect other user requirements concerning completeness and accuracy.
Investigating Generalisation in Continuous Deep Reinforcement Learning
Venue: arXiv cs.LG 2019
Year: 2019

Deep Reinforcement Learning has shown great success in a variety of control tasks. However, it is unclear how close we are to the vision of putting Deep RL into practice to solve real world problems. In particular, common practice in the field is to train policies on largely deterministic simulators and to evaluate algorithms through training performance alone, without a train/test distinction to ensure models generalise and are not overfitted. Moreover, it is not standard practice to check for generalisation under domain shift, although robustness to such system change between training and testing would be necessary for real-world Deep RL control, for example, in robotics. In this paper we study these issues by first characterising the sources of uncertainty that provide generalisation challenges in Deep RL. We then provide a new benchmark and thorough empirical evaluation of generalisation challenges for state of the art Deep RL methods. In particular, we show that, if generalisation is the goal, then common practice of evaluating algorithms based on their training performance leads to the wrong conclusions about algorithm choice. Finally, we evaluate several techniques for improving generalisation and draw conclusions about the most robust techniques to date.
Towards Touch-to-Access Device Authentication Using Induced Body Electric Potentials
Venue: arXiv cs.HC 2019
Year: 2019

This paper presents TouchAuth, a new touch-to-access device authentication approach using induced body electric potentials (iBEPs) caused by the indoor ambient electric field that is mainly emitted from the building's electrical cabling. The design of TouchAuth is based on the electrostatics of iBEP generation and a resulting property, i.e., the iBEPs at two close locations on the same human body are similar, whereas those from different human bodies are distinct. Extensive experiments verify the above property and show that TouchAuth achieves high-profile receiver operating characteristics in implementing the touch-to-access policy. Our experiments also show that a range of possible interfering sources including appliances' electromagnetic emanations and noise injections into the power network do not affect the performance of TouchAuth. A key advantage of TouchAuth is that the iBEP sensing requires a simple analog-to-digital converter only, which is widely available on microcontrollers. Compared with existing approaches including intra-body communication and physiological sensing, TouchAuth is a low-cost, lightweight, and convenient approach for authorized users to access the smart objects found in indoor environments.
Performance Analysis of Online Social Platforms
Venue: arXiv cs.NI 2019
Year: 2019

We introduce an original mathematical model to analyze the diffusion of posts within a generic online social platform. Each user of such a platform has his own Wall and Newsfeed, as well as his own self-posting and re-posting activity. As a main result, using our developed model, we derive in closed form the probabilities that posts originating from a given user are found on the Wall and Newsfeed of any other. These probabilities are the solution of a linear system of equations. Conditions of existence of the solution are provided, and two ways of solving the system are proposed, one using matrix inversion and another using fixed-point iteration. Comparisons with simulations show the accuracy of our model and its robustness with respect to the modeling assumptions. Hence, this article introduces a novel measure which allows to rank users by their influence on the social platform, by taking into account not only the social graph structure, but also the platform design, user activity (self- and re-posting), as well as competition among posts.
The Shutdown Problem: How Does a Blockchain System End?
Venue: arXiv cs.DC 2019
Year: 2019

We define and examine the shutdown problem for blockchain systems: how to gracefully end the system's operation at the end of its useful life. A particular focus is those blockchain systems that hold archival data of long-lived interest. We outline what it means to achieve a successful shutdown, and compare those criteria to likely end-of-life conditions in a generic blockchain system. We conclude that the decentralized nature of blockchain systems makes shutdown difficult, particularly if the system uses an unstable consensus like the Nakamoto consensus of Bitcoin. Accordingly, we recommend against using blockchain with unstable consensus for any data whose value is likely to persist beyond the life of the blockchain system. For any such systems that are already in operation, we recommend considering a hard fork to implement stable consensus. Such consideration needs to happen well in advance of the system's end of life.
Accurate Automatic Segmentation of Amygdala Subnuclei and Modeling of Uncertainty via Bayesian Fully Convolutional Neural Network
Venue: arXiv cs.CV 2019
Year: 2019

Recent advances in deep learning have improved the segmentation accuracy of subcortical brain structures, which would be useful in neuroimaging studies of many neurological disorders. However, most of the previous deep learning work does not investigate the specific difficulties that exist in segmenting extremely small but important brain regions such as the amygdala and its subregions. To tackle this challenging task, a novel 3D Bayesian fully convolutional neural network was developed to apply a dilated dualpathway approach that retains fine details and utilizes both local and more global contextual information to automatically segment the amygdala and its subregions at high precision. The proposed method provides insights on network design and sampling strategy that target segmentations of small 3D structures. In particular, this study confirms that a large context, enabled by a large field of view, is beneficial for segmenting small objects; furthermore, precise contextual information enabled by dilated convolutions allows for better boundary localization, which is critical for examining the morphology of the structure. In addition, it is demonstrated that the uncertainty information estimated from our network may be leveraged to identify atypicality in data. Our method was compared with two state-of-the-art deep learning models and a traditional multi-atlas approach, and exhibited excellent performance as measured both by Dice overlap as well as average symmetric surface distance. To the best of our knowledge, this work is the first deep learning-based approach that targets the subregions of the amygdala.
Extending the Anonymity of Zcash
Venue: arXiv cs.CR 2019
Year: 2019

Although Bitcoin in its original whitepaper stated that it offers anonymous transactions, de-anonymization techniques have found otherwise. Therefore, alternative cryptocurrencies, like Dash, Monero, and Zcash, were developed to provide better privacy. As Edward Snowden stated, "Zcash's privacy tech makes it the most interesting Bitcoin alternative (...) because the privacy properties of it are truly unique". Zcash's privacy is based on peer-reviewed cryptographic constructions, hence it is considered to provide the foundations for the best anonymity. However, even Zcash makes some privacy concessions. It does not protect users' privacy in the presence of a global adversary who is able to observe the whole network, and hence correlate the parties exchanging money, by using their network addresses. The recent empirical analysis of Zcash shows, that users often choose naive ways while performing the protocol operations, not realizing that it degrades their anonymity. In this talk, we will discuss an extension of Zcash using mix networks to enhance the privacy guarantees of users that choose to remain anonymous by tackling two major security challenges: one at the application layer of the scheme and one at its network layer.
An Autoencoder-based Learned Image Compressor: Description of Challenge Proposal by NCTU
Venue: arXiv cs.CV 2019
Year: 2019

We propose a lossy image compression system using the deep-learning autoencoder structure to participate in the Challenge on Learned Image Compression (CLIC) 2018. Our autoencoder uses the residual blocks with skip connections to reduce the correlation among image pixels and condense the input image into a set of feature maps, a compact representation of the original image. The bit allocation and bitrate control are implemented by using the importance maps and quantizer. The importance maps are generated by a separate neural net in the encoder. The autoencoder and the importance net are trained jointly based on minimizing a weighted sum of mean squared error, MS-SSIM, and a rate estimate. Our aim is to produce reconstructed images with good subjective quality subject to the 0.15 bits-per-pixel constraint.
Interpretation of Tactile Sensation using an Anthropomorphic Finger Motion Interface to Operate a Virtual Avatar
Venue: arXiv cs.HC 2019
Year: 2019

The objective of the system presented in this paper is to give users tactile feedback while walking in a virtual world through an anthropomorphic finger motion interface. We determined that the synchrony between the first person perspective and proprioceptive information together with the motor activity of the user's fingers are able to induce an illusionary feeling that is equivalent to the sense of ownership of the invisible avatar's legs. Under this condition, the perception of the ground under the virtual avatar's foot is felt through the user's fingertip. The experiments indicated that using our method the scale of the tactile perception of the texture roughness was extended and that the enlargement ratio was proportional to the avatar's body (foot) size. In order to display the target tactile perception to the users, we have to control only the virtual avatar's body (foot) size and the roughness of the tactile texture. Our results suggest that in terms of tactile perception fingers can be a replacement for legs in locomotion interfaces.
Learning Sparse Neural Networks via $\ell_0$ and T$\ell_1$ by a Relaxed Variable Splitting Method with Application to Multi-scale Curve Classification
Venue: arXiv cs.LG 2019
Year: 2019

We study sparsification of convolutional neural networks (CNN) by a relaxed variable splitting method of $\ell_0$ and transformed-$\ell_1$ (T$\ell_1$) penalties, with application to complex curves such as texts written in different fonts, and words written with trembling hands simulating those of Parkinson's disease patients. The CNN contains 3 convolutional layers, each followed by a maximum pooling, and finally a fully connected layer which contains the largest number of network weights. With $\ell_0$ penalty, we achieved over 99 \% test accuracy in distinguishing shaky vs. regular fonts or hand writings with above 86 \% of the weights in the fully connected layer being zero. Comparable sparsity and test accuracy are also reached with a proper choice of T$\ell_1$ penalty.
The Weak Call-By-Value λ-Calculus is Reasonable for Both Time and Space
Venue: arXiv cs.CC 2019
Year: 2019

We study the weak call-by-value $λ$-calculus as a model for computational complexity theory and establish the natural measures for time and space -- the number of beta-reductions and the size of the largest term in a computation -- as reasonable measures with respect to the invariance thesis of Slot and van Emde Boas [STOC~84]. More precisely, we show that, using those measures, Turing machines and the weak call-by-value $λ$-calculus can simulate each other within a polynomial overhead in time and a constant factor overhead in space for all computations that terminate in (encodings) of 'true' or 'false'. We consider this result as a solution to the long-standing open problem, explicitly posed by Accattoli [ENTCS~18], of whether the natural measures for time and space of the $λ$-calculus are reasonable, at least in case of weak call-by-value evaluation. Our proof relies on a hybrid of two simulation strategies of reductions in the weak call-by-value $λ$-calculus by Turing machines, both of which are insufficient if taken alone. The first strategy is the most naive one in the sense that a reduction sequence is simulated precisely as given by the reduction rules; in particular, all substitutions are executed immediately. This simulation runs within a constant overhead in space, but the overhead in time might be exponential. The second strategy is heap-based and relies on structure sharing, similar to existing compilers of eager functional languages. This strategy only has a polynomial overhead in time, but the space consumption might require an additional factor of $\log n$, which is essentially due to the size of the pointers required for this strategy. Our main contribution is the construction and verification of a space-aware interleaving of the two strategies, which is shown to yield both a constant overhead in space and a polynomial overhead in time.