OpenRaiser/Intern-Atlas huggingface dataset:
On the role of data in PAC-Bayes bounds
Venue: arXiv cs.LG 2020
Year: 2020

The dominant term in PAC-Bayes bounds is often the Kullback--Leibler divergence between the posterior and prior. For so-called linear PAC-Bayes risk bounds based on the empirical risk of a fixed posterior kernel, it is possible to minimize the expected value of the bound by choosing the prior to be the expected posterior, which we call the oracle prior on the account that it is distribution dependent. In this work, we show that the bound based on the oracle prior can be suboptimal: In some cases, a stronger bound is obtained by using a data-dependent oracle prior, i.e., a conditional expectation of the posterior, given a subset of the training data that is then excluded from the empirical risk term. While using data to learn a prior is a known heuristic, its essential role in optimal bounds is new. In fact, we show that using data can mean the difference between vacuous and nonvacuous bounds. We apply this new principle in the setting of nonconvex learning, simulating data-dependent oracle priors on MNIST and Fashion MNIST with and without held-out data, and demonstrating new nonvacuous bounds in both cases.
Convolutional Gaussian Embeddings for Personalized Recommendation with Uncertainty
Venue: arXiv cs.IR 2020
Year: 2020

Most of existing embedding based recommendation models use embeddings (vectors) corresponding to a single fixed point in low-dimensional space, to represent users and items. Such embeddings fail to precisely represent the users/items with uncertainty often observed in recommender systems. Addressing this problem, we propose a unified deep recommendation framework employing Gaussian embeddings, which are proven adaptive to uncertain preferences exhibited by some users, resulting in better user representations and recommendation performance. Furthermore, our framework adopts Monte-Carlo sampling and convolutional neural networks to compute the correlation between the objective user and the candidate item, based on which precise recommendations are achieved. Our extensive experiments on two benchmark datasets not only justify that our proposed Gaussian embeddings capture the uncertainty of users very well, but also demonstrate its superior performance over the state-of-the-art recommendation models.
Bayesian Optimization with Missing Inputs
Venue: arXiv cs.LG 2020
Year: 2020

Bayesian optimization (BO) is an efficient method for optimizing expensive black-box functions. In real-world applications, BO often faces a major problem of missing values in inputs. The missing inputs can happen in two cases. First, the historical data for training BO often contain missing values. Second, when performing the function evaluation (e.g. computing alloy strength in a heat treatment process), errors may occur (e.g. a thermostat stops working) leading to an erroneous situation where the function is computed at a random unknown value instead of the suggested value. To deal with this problem, a common approach just simply skips data points where missing values happen. Clearly, this naive method cannot utilize data efficiently and often leads to poor performance. In this paper, we propose a novel BO method to handle missing inputs. We first find a probability distribution of each missing value so that we can impute the missing value by drawing a sample from its distribution. We then develop a new acquisition function based on the well-known Upper Confidence Bound (UCB) acquisition function, which considers the uncertainty of imputed values when suggesting the next point for function evaluation. We conduct comprehensive experiments on both synthetic and real-world applications to show the usefulness of our method.
Attention Mesh: High-fidelity Face Mesh Prediction in Real-time
Venue: arXiv cs.CV 2020
Year: 2020

We present Attention Mesh, a lightweight architecture for 3D face mesh prediction that uses attention to semantically meaningful regions. Our neural network is designed for real-time on-device inference and runs at over 50 FPS on a Pixel 2 phone. Our solution enables applications like AR makeup, eye tracking and AR puppeteering that rely on highly accurate landmarks for eye and lips regions. Our main contribution is a unified network architecture that achieves the same accuracy on facial landmarks as a multi-stage cascaded approach, while being 30 percent faster.
Monash University, UEA, UCR Time Series Extrinsic Regression Archive
Venue: arXiv cs.LG 2020
Year: 2020

Time series research has gathered lots of interests in the last decade, especially for Time Series Classification (TSC) and Time Series Forecasting (TSF). Research in TSC has greatly benefited from the University of California Riverside and University of East Anglia (UCR/UEA) Time Series Archives. On the other hand, the advancement in Time Series Forecasting relies on time series forecasting competitions such as the Makridakis competitions, NN3 and NN5 Neural Network competitions, and a few Kaggle competitions. Each year, thousands of papers proposing new algorithms for TSC and TSF have utilized these benchmarking archives. These algorithms are designed for these specific problems, but may not be useful for tasks such as predicting the heart rate of a person using photoplethysmogram (PPG) and accelerometer data. We refer to this problem as Time Series Extrinsic Regression (TSER), where we are interested in a more general methodology of predicting a single continuous value, from univariate or multivariate time series. This prediction can be from the same time series or not directly related to the predictor time series and does not necessarily need to be a future value or depend heavily on recent values. To the best of our knowledge, research into TSER has received much less attention in the time series research community and there are no models developed for general time series extrinsic regression problems. Most models are developed for a specific problem. Therefore, we aim to motivate and support the research into TSER by introducing the first TSER benchmarking archive. This archive contains 19 datasets from different domains, with varying number of dimensions, unequal length dimensions, and missing values. In this paper, we introduce the datasets in this archive and did an initial benchmark on existing models.
Adversarial Attacks for Multi-view Deep Models
Venue: arXiv cs.LG 2020
Year: 2020

Recent work has highlighted the vulnerability of many deep machine learning models to adversarial examples. It attracts increasing attention to adversarial attacks, which can be used to evaluate the security and robustness of models before they are deployed. However, to our best knowledge, there is no specific research on the adversarial attacks for multi-view deep models. This paper proposes two multi-view attack strategies, two-stage attack (TSA) and end-to-end attack (ETEA). With the mild assumption that the single-view model on which the target multi-view model is based is known, we first propose the TSA strategy. The main idea of TSA is to attack the multi-view model with adversarial examples generated by attacking the associated single-view model, by which state-of-the-art single-view attack methods are directly extended to the multi-view scenario. Then we further propose the ETEA strategy when the multi-view model is provided publicly. The ETEA is applied to accomplish direct attacks on the target multi-view model, where we develop three effective multi-view attack methods. Finally, based on the fact that adversarial examples generalize well among different models, this paper takes the adversarial attack on the multi-view convolutional neural network as an example to validate that the effectiveness of the proposed multi-view attacks. Extensive experimental results demonstrate that our multi-view attack strategies are capable of attacking the multi-view deep models, and we additionally find that multi-view models are more robust than single-view models.
Design of a Near-Ideal Fault-Tolerant Routing Algorithm for Network-on-Chip-Based Multicores
Venue: arXiv cs.AR 2020
Year: 2020

With relentless CMOS technology downsizing Networks-on-Chips (NoCs) are inescapably experiencing escalating susceptibility to wearout and reduced reliability. While faults in processors and memories may be masked via redundancy, or mitigated via techniques such as task migration, NoCs are especially vulnerable to hardware faults as a single link breakdown may cause inter-tile communication to halt indefinitely, rendering the whole multicore chip inoperable. As such, NoCs impose the risk of becoming the pivotal point of failure in chip multicores that utilize them. Aiming towards seamless NoC operation in the presence of faulty links we propose Hermes, a near-ideal fault-tolerant routing algorithm that meets the objectives of exhibiting high levels of robustness, operating in a distributed mode, guaranteeing freedom from deadlocks, and evening-out traffic, among many. Hermes is a limited-overhead deadlock-free hybrid routing algorithm, utilizing load-balancing routing on fault-free paths to sustain high-throughput, while providing pre-reconfigured escape path selection in the vicinity of faults. Under such online mechanisms, Hermes's performance degrades gracefully with increasing faulty link counts, a crucially desirable response lacking in prior-art. Additionally, Hermes identifies non-communicating network partitions in scenarios where faulty links are topologically densely distributed such that packets being routed to physically isolated regions cause no network stagnation due to indefinite chained blockages starting at sub-network boundaries. An extensive experimental evaluation, including utilizing traffic workloads gathered from full-system chip multi-processor simulations, shows that Hermes improves network throughput by up to $3\times$ when compared against the state-of-the-art. Further, hardware synthesis results prove Hermes's efficacy.
Hybridizing the 1/5-th Success Rule with Q-Learning for Controlling the Mutation Rate of an Evolutionary Algorithm
Venue: arXiv cs.NE 2020
Year: 2020

It is well known that evolutionary algorithms (EAs) achieve peak performance only when their parameters are suitably tuned to the given problem. Even more, it is known that the best parameter values can change during the optimization process. Parameter control mechanisms are techniques developed to identify and to track these values. Recently, a series of rigorous theoretical works confirmed the superiority of several parameter control techniques over EAs with best possible static parameters. Among these results are examples for controlling the mutation rate of the $(1+λ)$~EA when optimizing the OneMax problem. However, it was shown in [Rodionova et al., GECCO'19] that the quality of these techniques strongly depends on the offspring population size $λ$. We introduce in this work a new hybrid parameter control technique, which combines the well-known one-fifth success rule with Q-learning. We demonstrate that our HQL mechanism achieves equal or superior performance to all techniques tested in [Rodionova et al., GECCO'19] and this -- in contrast to previous parameter control methods -- simultaneously for all offspring population sizes $λ$. We also show that the promising performance of HQL is not restricted to OneMax, but extends to several other benchmark problems.
On the Principle of Accountability: Challenges for Smart Homes & Cybersecurity
Venue: arXiv cs.HC 2020
Year: 2020

This chapter introduces the Accountability Principle and its role in data protection governance. We focus on what accountability means in the context of cybersecurity management in smart homes, considering the EU General Data Protection Law requirements to secure personal data. This discussion sits against the backdrop of two key new developments in data protection law. Firstly, the law is moving into the home, due to narrowing of the so called household exemption. Concurrently, household occupants may now have legal responsibilities to comply with the GDPR, as they find themselves jointly responsible for compliance, as they are possibly held to determine the means and purposes of data collection with IoT device vendors. As a complex socio-technical space, we consider the interactions between accountability requirements and the competencies of this new class of domestic data controllers (DDCs). Specifically, we consider the value and limitations of edge-based security analytics to manage smart home cybersecurity risks, reviewing a range of prototypes and studies of their use. We also reflect on interpersonal power dynamics in the domestic setting e.g. device control; existing social practices around privacy and security management in smart homes; and usability issues that may hamper DDCs ability to rely on such solutions. We conclude by reflecting on 1) the need for collective security management in homes and 2) the increasingly complex divisions of responsibility in smart homes between device users, account holders, IoT device/software/firmware vendors, and third parties.
Artificial Buildings: Safety, Complexity and a Quantifiable Measure of Beauty
Venue: arXiv cs.OH 2020
Year: 2020

A place to live is one of the most crucial necessities for all living organisms since the advent of life on planet Earth. The nature of homes has changed considerably over time. At the very early stages, human begins lived in natural places such as caves. Later on, they started to use their intelligence to build places with special purposes. Nowadays, modern technologies such as robotics and artificial intelligence have made their ways into the construction process and opened up a whole new area of opportunities and concerns that may be of interest to both technologists and philosophers. In this article, I review the evolution of buildings from fully natural to fully artificial and discuss philosophical thoughts that a fully automated construction technology may raise. I elaborate on the safety concerns of a fully automated architectural process. Then, I'll borrow Kolmogorov complexity from algorithmic information theory to define a complexity measure for buildings. The proposed measure is then used to provide a quantifiable measure of beauty.
All you can stream: Investigating the role of user behavior for greenhouse gas intensity of video streaming
Venue: arXiv cs.CY 2020
Year: 2020

The information and communication technology sector reportedly has a relevant impact on the environment. Within this sector, video streaming has been identified as a major driver of CO2-emissions. To make streaming more sustainable, environmentally relevant factors must be identified on both the user and the provider side. Hence, environmental assessments, like life cycle assessments (LCA), need to broaden their perspective from a mere technological to one that includes user decisions and behavior. However, quantitative data on user behavior (e.g. streaming duration, choice of end device and resolution) are often lacking or difficult to integrate in LCA. Additionally, identifying relevant determinants of user behavior, such as the design of streaming platforms or user motivations, may help to design streaming services that keep environmental impact at a passable level. In order to carry out assessments in such a way, interdisciplinary collaboration is necessary. Therefore, this exploratory study combined LCA with an online survey (N= 91, 7 consecutive days of assessment). Based on this dataset the use phase of online video streaming was modeled. Additionally, factors such as sociodemographic, motivational and contextual determinants were measured. Results show that CO2-intensity of video streaming depends on several factors. It is shown that for climate intensity there is a factor 10 between choosing a smart TV and smartphone for video streaming. Furthermore, results show that some factors can be tackled from provider side to reduce overall energy demand at the user side; one of which is setting a low resolution as default.
List decoding of Convolutional Codes over integer residue rings
Venue: arXiv cs.IT 2020
Year: 2020

A convolutional code $\C$ over $\ZZ[D]$ is a $\ZZ[D]$-submodule of $\ZZN[D]$ where $\ZZ[D]$ stands for the ring of polynomials with coefficients in $\ZZ$. In this paper, we study the list decoding problem of these codes when the transmission is performed over an erasure channel, that is, we study how much information one can recover from a codeword $w\in \C$ when some of its coefficients have been erased. We do that using the $p$-adic expansion of $w$ and particular representations of the parity-check polynomial matrix of the code. From these matrix polynomial representations we recursively select certain equations that $w$ must satisfy and have only coefficients in the field $p^{r-1}\ZZ$. We exploit the natural block Toeplitz structure of the sliding parity-check matrix to derive a step by step methodology to obtain a list of possible codewords for a given corrupted codeword $w$, that is, a list with the closest codewords to $w$.
Modelling Agent Policies with Interpretable Imitation Learning
Venue: arXiv cs.AI 2020
Year: 2020

As we deploy autonomous agents in safety-critical domains, it becomes important to develop an understanding of their internal mechanisms and representations. We outline an approach to imitation learning for reverse-engineering black box agent policies in MDP environments, yielding simplified, interpretable models in the form of decision trees. As part of this process, we explicitly model and learn agents' latent state representations by selecting from a large space of candidate features constructed from the Markov state. We present initial promising results from an implementation in a multi-agent traffic environment.
Image Sentiment Transfer
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we introduce an important but still unexplored research task -- image sentiment transfer. Compared with other related tasks that have been well-studied, such as image-to-image translation and image style transfer, transferring the sentiment of an image is more challenging. Given an input image, the rule to transfer the sentiment of each contained object can be completely different, making existing approaches that perform global image transfer by a single reference image inadequate to achieve satisfactory performance. In this paper, we propose an effective and flexible framework that performs image sentiment transfer at the object level. It first detects the objects and extracts their pixel-level masks, and then performs object-level sentiment transfer guided by multiple reference images for the corresponding objects. For the core object-level sentiment transfer, we propose a novel Sentiment-aware GAN (SentiGAN). Both global image-level and local object-level supervisions are imposed to train SentiGAN. More importantly, an effective content disentanglement loss cooperating with a content alignment step is applied to better disentangle the residual sentiment-related information of the input image. Extensive quantitative and qualitative experiments are performed on the object-oriented VSO dataset we create, demonstrating the effectiveness of the proposed framework.
Student Mixture Model Based Visual Servoing
Venue: arXiv cs.RO 2020
Year: 2020

Classical Image-Based Visual Servoing (IBVS) makes use of geometric image features like point, straight line and image moments to control a robotic system. Robust extraction and real-time tracking of these features are crucial to the performance of the IBVS. Moreover, such features can be unsuitable for real world applications where it might not be easy to distinguish a target from the rest of the environment. Alternatively, an approach based on complete photometric data can avoid the requirement of feature extraction, tracking and object detection. In this work, we propose one such probabilistic model based approach which uses entire photometric data for the purpose of visual servoing. A novel image modelling method has been proposed using Student Mixture Model (SMM), which is based on Multivariate Student's t-Distribution. Consequently, a vision-based control law is formulated as a least squares minimisation problem. Efficacy of the proposed framework is demonstrated for 2D and 3D positioning tasks showing favourable error convergence and acceptable camera trajectories. Numerical experiments are also carried out to show robustness to distinct image scenes and partial occlusion.
Bombus Species Image Classification
Venue: arXiv cs.CV 2020
Year: 2020

Entomologists, ecologists and others struggle to rapidly and accurately identify the species of bumble bees they encounter in their field work and research. The current process requires the bees to be mounted, then physically shipped to a taxonomic expert for proper categorization. We investigated whether an image classification system derived from transfer learning can do this task. We used Google Inception, Oxford VGG16 and VGG19 and Microsoft ResNet 50. We found Inception and VGG classifiers were able to make some progress at identifying bumble bee species from the available data, whereas ResNet was not. Individual classifiers achieved accuracies of up to 23% for single species identification and 44% top-3 labels, where a composite model performed better, 27% and 50%. We feel the performance was most hampered by our limited data set of 5,000-plus labeled images of 29 species, with individual species represented by 59 -315 images.
Sparsifying and Down-scaling Networks to Increase Robustness to Distortions
Venue: arXiv cs.CV 2020
Year: 2020

It has been shown that perfectly trained networks exhibit drastic reduction in performance when presented with distorted images. Streaming Network (STNet) is a novel architecture capable of robust classification of the distorted images while been trained on undistorted images. The distortion robustness is enabled by means of sparse input and isolated parallel streams with decoupled weights. Recent results prove STNet is robust to 20 types of noise and distortions. STNet exhibits state-of-the-art performance for classification of low light images, while being of much smaller size when other networks. In this paper, we construct STNets by using scaled versions (number of filters in each layer is reduced by factor of n) of popular networks like VGG16, ResNet50 and MobileNetV2 as parallel streams. These new STNets are tested on several datasets. Our results indicate that more efficient (less FLOPS), new STNets exhibit higher or equal accuracy in comparison with original networks. Considering a diversity of datasets and networks used for tests, we conclude that a new type of STNets is an efficient tool for robust classification of distorted images.
Unifying Few- and Zero-Shot Egocentric Action Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Although there has been significant research in egocentric action recognition, most methods and tasks, including EPIC-KITCHENS, suppose a fixed set of action classes. Fixed-set classification is useful for benchmarking methods, but is often unrealistic in practical settings due to the compositionality of actions, resulting in a functionally infinite-cardinality label set. In this work, we explore generalization with an open set of classes by unifying two popular approaches: few- and zero-shot generalization (the latter which we reframe as cross-modal few-shot generalization). We propose a new set of splits derived from the EPIC-KITCHENS dataset that allow evaluation of open-set classification, and use these splits to show that adding a metric-learning loss to the conventional direct-alignment baseline can improve zero-shot classification by as much as 10%, while not sacrificing few-shot performance.