OpenRaiser/Intern-Atlas huggingface dataset:
The Trusted Edge
Venue: arXiv cs.DC 2021
Year: 2021

Edge computing promises to reshape the centralized nature of today's cloud-based applications by bringing computing resources, at least in part, closer to the user. Reasons include the increasing need for real-time (short-delay, reliably-connected) computing and resource-demanding artificial intelligence (AI) algorithms that overstrain mobile devices' batteries or compute power but are too bandwidth-demanding to be offloaded to a distant cloud. However, companies may need to run their protected business logic on (untrusted) third-party edge devices, which can lead to serious issues due to weaker security measures than in cloud environments. This article makes the case for trusted edge computing (TEC), which focuses on developing concepts and methods for protecting application providers' business logic (and thus their intellectual property) specifically tailored to open edge infrastructures. This article further discusses open challenges in TEC to be addressed in the future. Otherwise, edge computing risks being a non-starter for most businesses due to the inadequate and neglected protection of intellectual property.
Not Far Away, Not So Close: Sample Efficient Nearest Neighbour Data Augmentation via MiniMax
Venue: arXiv cs.CL 2021
Year: 2021

In Natural Language Processing (NLP), finding data augmentation techniques that can produce high-quality human-interpretable examples has always been challenging. Recently, leveraging kNN such that augmented examples are retrieved from large repositories of unlabelled sentences has made a step toward interpretable augmentation. Inspired by this paradigm, we introduce Minimax-kNN, a sample efficient data augmentation strategy tailored for Knowledge Distillation (KD). We exploit a semi-supervised approach based on KD to train a model on augmented data. In contrast to existing kNN augmentation techniques that blindly incorporate all samples, our method dynamically selects a subset of augmented samples that maximizes KL-divergence between the teacher and student models. This step aims to extract the most efficient samples to ensure our augmented data covers regions in the input space with maximum loss value. We evaluated our technique on several text classification tasks and demonstrated that Minimax-kNN consistently outperforms strong baselines. Our results show that Minimax-kNN requires fewer augmented examples and less computation to achieve superior performance over the state-of-the-art kNN-based augmentation techniques.
Optimal Model Placement and Online Model Splitting for Device-Edge Co-Inference
Venue: arXiv cs.LG 2021
Year: 2021

Device-edge co-inference opens up new possibilities for resource-constrained wireless devices (WDs) to execute deep neural network (DNN)-based applications with heavy computation workloads. In particular, the WD executes the first few layers of the DNN and sends the intermediate features to the edge server that processes the remaining layers of the DNN. By adapting the model splitting decision, there exists a tradeoff between local computation cost and communication overhead. In practice, the DNN model is re-trained and updated periodically at the edge server. Once the DNN parameters are regenerated, part of the updated model must be placed at the WD to facilitate on-device inference. In this paper, we study the joint optimization of the model placement and online model splitting decisions to minimize the energy-and-time cost of device-edge co-inference in presence of wireless channel fading. The problem is challenging because the model placement and model splitting decisions are strongly coupled, while involving two different time scales. We first tackle online model splitting by formulating an optimal stopping problem, where the finite horizon of the problem is determined by the model placement decision. In addition to deriving the optimal model splitting rule based on backward induction, we further investigate a simple one-stage look-ahead rule, for which we are able to obtain analytical expressions of the model splitting decision. The analysis is useful for us to efficiently optimize the model placement decision in a larger time scale. In particular, we obtain a closed-form model placement solution for the fully-connected multilayer perceptron with equal neurons. Simulation results validate the superior performance of the joint optimal model placement and splitting with various DNN structures.
AdvParams: An Active DNN Intellectual Property Protection Technique via Adversarial Perturbation Based Parameter Encryption
Venue: arXiv cs.CR 2021
Year: 2021

A well-trained DNN model can be regarded as an intellectual property (IP) of the model owner. To date, many DNN IP protection methods have been proposed, but most of them are watermarking based verification methods where model owners can only verify their ownership passively after the copyright of DNN models has been infringed. In this paper, we propose an effective framework to actively protect the DNN IP from infringement. Specifically, we encrypt the DNN model's parameters by perturbing them with well-crafted adversarial perturbations. With the encrypted parameters, the accuracy of the DNN model drops significantly, which can prevent malicious infringers from using the model. After the encryption, the positions of encrypted parameters and the values of the added adversarial perturbations form a secret key. Authorized user can use the secret key to decrypt the model. Compared with the watermarking methods which only passively verify the ownership after the infringement occurs, the proposed method can prevent infringement in advance. Moreover, compared with most of the existing active DNN IP protection methods, the proposed method does not require additional training process of the model, which introduces low computational overhead. Experimental results show that, after the encryption, the test accuracy of the model drops by 80.65%, 81.16%, and 87.91% on Fashion-MNIST, CIFAR-10, and GTSRB, respectively. Moreover, the proposed method only needs to encrypt an extremely low number of parameters, and the proportion of the encrypted parameters of all the model's parameters is as low as 0.000205%. The experimental results also indicate that, the proposed method is robust against model fine-tuning attack and model pruning attack. Moreover, for the adaptive attack where attackers know the detailed steps of the proposed method, the proposed method is also demonstrated to be robust.
Embedded Vision for Self-Driving on Forest Roads
Venue: arXiv cs.CV 2021
Year: 2021

Forest roads in Romania are unique natural wildlife sites used for recreation by countless tourists. In order to protect and maintain these roads, we propose RovisLab AMTU (Autonomous Mobile Test Unit), which is a robotic system designed to autonomously navigate off-road terrain and inspect if any deforestation or damage occurred along tracked route. AMTU's core component is its embedded vision module, optimized for real-time environment perception. For achieving a high computation speed, we use a learning system to train a multi-task Deep Neural Network (DNN) for scene and instance segmentation of objects, while the keypoints required for simultaneous localization and mapping are calculated using a handcrafted FAST feature detector and the Lucas-Kanade tracking algorithm. Both the DNN and the handcrafted backbone are run in parallel on the GPU of an NVIDIA AGX Xavier board. We show experimental results on the test track of our research facility.
Mapping urban socioeconomic inequalities in developing countries through Facebook advertising data
Venue: arXiv cs.CY 2021
Year: 2021

Ending poverty in all its forms everywhere is the number one Sustainable Development Goal of the UN 2030 Agenda. To monitor the progress towards such an ambitious target, reliable, up-to-date and fine-grained measurements of socioeconomic indicators are necessary. When it comes to socioeconomic development, novel digital traces can provide a complementary data source to overcome the limits of traditional data collection methods, which are often not regularly updated and lack adequate spatial resolution. In this study, we collect publicly available and anonymous advertising audience estimates from Facebook to predict socioeconomic conditions of urban residents, at a fine spatial granularity, in four large urban areas: Atlanta (USA), Bogotá (Colombia), Santiago (Chile), and Casablanca (Morocco). We find that behavioral attributes inferred from the Facebook marketing platform can accurately map the socioeconomic status of residential areas within cities, and that predictive performance is comparable in both high and low-resource settings. We also show that training a model on attributes of adult Facebook users, aged more than 25, leads to a more accurate mapping of socioeconomic conditions in all cities. Our work provides additional evidence of the value of social advertising media data to measure human development.
Linear Complexity of Binary Interleaved Sequences of Period 4n
Venue: arXiv cs.IT 2021
Year: 2021

Binary periodic sequences with good autocorrelation property have many applications in many aspects of communication. In past decades many series of such binary sequences have been constructed. In the application of cryptography, such binary sequences are required to have larger linear complexity. Tang and Ding \cite{X. Tang} presented a method to construct a series of binary sequences with period 4$n$ having optimal autocorrelation. Such sequences are interleaved by two arbitrary binary sequences with period $n\equiv 3\pmod 4$ and ideal autocorrelation. In this paper we present a general formula on the linear complexity of such interleaved sequences. Particularly, we show that the linear complexity of such sequences with period 4$n$ is not bigger than $2n+2$. Interleaving by several types of known binary sequences with ideal autocorrelation ($m$-sequences, Legendre, twin-prime and Hall's sequences), we present many series of such sequences having the maximum value $2n+2$ of linear complexity which gives an answer of a problem raised by N. Li and X. Tang \cite{N. Li}. Finally, in the conclusion section we show that it can be seen easily that the 2-adic complexity of all such interleaved sequences reaches the maximum value $\log_{2}(2^{4n}-1)$.
Do not explain without context: addressing the blind spot of model explanations
Venue: arXiv cs.LG 2021
Year: 2021

The increasing number of regulations and expectations of predictive machine learning models, such as so called right to explanation, has led to a large number of methods promising greater interpretability. High demand has led to a widespread adoption of XAI techniques like Shapley values, Partial Dependence profiles or permutational variable importance. However, we still do not know enough about their properties and how they manifest in the context in which explanations are created by analysts, reviewed by auditors, and interpreted by various stakeholders. This paper highlights a blind spot which, although critical, is often overlooked when monitoring and auditing machine learning models: the effect of the reference data on the explanation calculation. We discuss that many model explanations depend directly or indirectly on the choice of the referenced data distribution. We showcase examples where small changes in the distribution lead to drastic changes in the explanations, such as a change in trend or, alarmingly, a conclusion. Consequently, we postulate that obtaining robust and useful explanations always requires supporting them with a broader context.
The Herbarium 2021 Half-Earth Challenge Dataset
Venue: arXiv cs.CV 2021
Year: 2021

Herbarium sheets present a unique view of the world's botanical history, evolution, and diversity. This makes them an all-important data source for botanical research. With the increased digitisation of herbaria worldwide and the advances in the fine-grained classification domain that can facilitate automatic identification of herbarium specimens, there are a lot of opportunities for supporting research in this field. However, existing datasets are either too small, or not diverse enough, in terms of represented taxa, geographic distribution or host institutions. Furthermore, aggregating multiple datasets is difficult as taxa exist under a multitude of different names and the taxonomy requires alignment to a common reference. We present the Herbarium Half-Earth dataset, the largest and most diverse dataset of herbarium specimens to date for automatic taxon recognition.
SEVerity: Code Injection Attacks against Encrypted Virtual Machines
Venue: arXiv cs.CR 2021
Year: 2021

Modern enterprises increasingly take advantage of cloud infrastructures. Yet, outsourcing code and data into the cloud requires enterprises to trust cloud providers not to meddle with their data. To reduce the level of trust towards cloud providers, AMD has introduced Secure Encrypted Virtualization (SEV). By encrypting Virtual Machines (VMs), SEV aims to ensure data confidentiality, despite a compromised or curious Hypervisor. The SEV Encrypted State (SEV-ES) extension additionally protects the VM's register state from unauthorized access. Yet, both extensions do not provide integrity of the VM's memory, which has already been abused to leak the protected data or to alter the VM's control-flow. In this paper, we introduce the SEVerity attack; a missing puzzle piece in the series of attacks against the AMD SEV family. Specifically, we abuse the system's lack of memory integrity protection to inject and execute arbitrary code within SEV-ES-protected VMs. Contrary to previous code execution attacks against the AMD SEV family, SEVerity neither relies on a specific CPU version nor on any code gadgets inside the VM. Instead, SEVerity abuses the fact that SEV-ES prohibits direct memory access into the encrypted memory. Specifically, SEVerity injects arbitrary code into the encrypted VM through I/O channels and uses the Hypervisor to locate and trigger the execution of the encrypted payload. This allows us to sidestep the protection mechanisms of SEV-ES. Overall, our results demonstrate a success rate of 100% and hence highlight that memory integrity protection is an obligation when encrypting VMs. Consequently, our work presents the final stroke in a series of attacks against AMD SEV and SEV-ES and renders the present implementation as incapable of protecting against a curious, vulnerable, or malicious Hypervisor.
Regret-Minimizing Bayesian Persuasion
Venue: arXiv cs.GT 2021
Year: 2021

We study a Bayesian persuasion setting with binary actions (adopt and reject) for Receiver. We examine the following question - how well can Sender perform, in terms of persuading Receiver to adopt, when ignorant of Receiver's utility? We take a robust (adversarial) approach to study this problem; that is, our goal is to design signaling schemes for Sender that perform well for all possible Receiver's utilities. We measure performance of signaling schemes via the notion of (additive) regret: the difference between Sender's hypothetically optimal utility had she known Receiver's utility function and her actual utility induced by the given scheme. On the negative side, we show that if Sender has no knowledge at all about Receiver's utility, then Sender has no signaling scheme that performs robustly well. On the positive side, we show that if Sender only knows Receiver's ordinal preferences of the states of nature - i.e., Receiver's utility upon adoption is monotonic as a function of the state - then Sender can guarantee a surprisingly low regret even when the number of states tends to infinity. In fact, we exactly pin down the minimum regret value that Sender can guarantee in this case, which turns out to be at most 1/e. We further show that such positive results are not possible under the alternative performance measure of a multiplicative approximation ratio by proving that no constant ratio can be guaranteed even for monotonic Receiver's utility; this may serve to demonstrate the merits of regret as a robust performance measure that is not too pessimistic. Finally, we analyze an intermediate setting in between the no-knowledge and the ordinal-knowledge settings.
Capacity of Backscatter Communication under Arbitrary Fading Dependence
Venue: arXiv cs.IT 2021
Year: 2021

We analyze the impact of arbitrary dependence between the forward and backward links in backscatter communication systems. Specifically, we quantify the effect of positive and negative dependence between these fading links on channel capacity, using Copula theory. The benefits of this approach are highlighted over the classical framework of linear dependence based on Pearson's correlation coefficient, which is also analyzed. Results show that for a fixed transmit power budget, capacity grows with positive dependence as well as with fading severity in the low signal-to-noise ratio (SNR) regime. Conversely, fading dependence becomes immaterial in the high SNR regime.
Finite-Horizon LQR Control of Quadrotors on $SE_2(3)$
Venue: arXiv cs.RO 2021
Year: 2021

This paper considers optimal control of a quadrotor unmanned aerial vehicles (UAV) using the discrete-time, finite-horizon, linear quadratic regulator (LQR). The state of a quadrotor UAV is represented as an element of the matrix Lie group of double direct isometries, $SE_2(3)$. The nonlinear system is linearized using a left-invariant error about a reference trajectory, leading to an optimal gain sequence that can be calculated offline. The reference trajectory is calculated using the differentially flat properties of the quadrotor. Monte-Carlo simulations demonstrate robustness of the proposed control scheme to parametric uncertainty, state-estimation error, and initial error. Additionally, when compared to an LQR controller that uses a conventional error definition, the proposed controller demonstrates better performance when initial errors are large.
Reflections on Termination of Linear Loops
Venue: arXiv cs.PL 2021
Year: 2021

This paper shows how techniques for linear dynamical systems can be used to reason about the behavior of general loops. We present two main results. First, we show that every loop that can be expressed as a transition formula in linear integer arithmetic has a best model as a deterministic affine transition system. Second, we show that for any linear dynamical system $f$ with integer eigenvalues and any integer arithmetic formula $G$, there is a linear integer arithmetic formula that holds exactly for the states of $f$ for which $G$ is eventually invariant. Combining the two, we develop a monotone conditional termination analysis for general loops.
The Evaluation of Rating Systems in Team-based Battle Royale Games
Venue: arXiv cs.IR 2021
Year: 2021

Online competitive games have become a mainstream entertainment platform. To create a fair and exciting experience, these games use rating systems to match players with similar skills. While there has been an increasing amount of research on improving the performance of these systems, less attention has been paid to how their performance is evaluated. In this paper, we explore the utility of several metrics for evaluating three popular rating systems on a real-world dataset of over 25,000 team battle royale matches. Our results suggest considerable differences in their evaluation patterns. Some metrics were highly impacted by the inclusion of new players. Many could not capture the real differences between certain groups of players. Among all metrics studied, normalized discounted cumulative gain (NDCG) demonstrated more reliable performance and more flexibility. It alleviated most of the challenges faced by the other metrics while adding the freedom to adjust the focus of the evaluations on different groups of players.
Cloud Collectives: Towards Cloud-aware Collectives forML Workloads with Rank Reordering
Venue: arXiv cs.DC 2021
Year: 2021

ML workloads are becoming increasingly popular in the cloud. Good cloud training performance is contingent on efficient parameter exchange among VMs. We find that Collectives, the widely used distributed communication algorithms, cannot perform optimally out of the box due to the hierarchical topology of datacenter networks and multi-tenancy nature of the cloudenvironment.In this paper, we present Cloud Collectives , a prototype that accelerates collectives by reordering theranks of participating VMs such that the communication pattern dictated by the selected collectives operation best exploits the locality in the network.Collectives is non-intrusive, requires no code changes nor rebuild of an existing application, and runs without support from cloud providers. Our preliminary application of Cloud Collectives on allreduce operations in public clouds results in a speedup of up to 3.7x in multiple microbenchmarks and 1.3x in real-world workloads of distributed training of deep neural networks and gradient boosted decision trees using state-of-the-art frameworks.
Towards a modeling and analysis environment for industrial IoT systems
Venue: arXiv cs.SE 2021
Year: 2021

The development of Industrial Internet of Things systems (IIoT) requires tools robust enough to cope with the complexity and heterogeneity of such systems, which are supposed to work in safety-critical conditions. The availability of methodologies to support early analysis, verification, and validation is still an open issue in the research community. The early real-time schedulability analysis can help quantify to what extent the desired system's timing performance can eventually be achieved. In this paper, we present CHESSIoT, a model-driven environment to support the design and analysis of industrial IoT systems. CHESSIoT follows a multi-view, component-based modelling approach with a comprehensive way to perform event-based modelling on system components for code generation purposes employing an intermediate ThingML model. To showcase the capability of the extension, we have designed and analysed an Industrial real-time safety use case.
SMASH: Sparse Matrix Atomic Scratchpad Hashing
Venue: arXiv cs.DC 2021
Year: 2021

Sparse matrices, more specifically SpGEMM kernels, are commonly found in a wide range of applications, spanning graph-based path-finding to machine learning algorithms (e.g., neural networks). A particular challenge in implementing SpGEMM kernels has been the pressure placed on DRAM memory. One approach to tackle this problem is to use an inner product method for the SpGEMM kernel implementation. While the inner product produces fewer intermediate results, it can end up saturating the memory bandwidth, given the high number of redundant fetches of the input matrix elements. Using an outer product-based SpGEMM kernel can reduce redundant fetches, but at the cost of increased overhead due to extra computation and memory accesses for producing/managing partial products. In this thesis, we introduce a novel SpGEMM kernel implementation based on the row-wise product approach. We leverage atomic instructions to merge intermediate partial products as they are generated. The use of atomic instructions eliminates the need to create partial product matrices. To evaluate our row-wise product approach, we map an optimized SpGEMM kernel to a custom accelerator designed to accelerate graph-based applications. The targeted accelerator is an experimental system named PIUMA, being developed by Intel. PIUMA provides several attractive features, including fast context switching, user-configurable caches, globally addressable memory, non-coherent caches, and asynchronous pipelines. We tailor our SpGEMM kernel to exploit many of the features of the PIUMA fabric. This thesis compares our SpGEMM implementation against prior solutions, all mapped to the PIUMA framework. We briefly describe some of the PIUMA architecture features and then delve into the details of our optimized SpGEMM kernel. Our SpGEMM kernel can achieve 9.4x speedup as compared to competing approaches.