OpenRaiser/Intern-Atlas huggingface dataset:
Battery-constrained Federated Edge Learning in UAV-enabled IoT for B5G/6G Networks
Venue: arXiv cs.NI 2021
Year: 2021

In this paper, we study how to optimize the federated edge learning (FEEL) in UAV-enabled Internet of things (IoT) for B5G/6G networks, from a deep reinforcement learning (DRL) approach. The federated learning is an effective framework to train a shared model between decentralized edge devices or servers without exchanging raw data, which can help protect data privacy. In UAV-enabled IoT networks, latency and energy consumption are two important metrics limiting the performance of FEEL. Although most of existing works have studied how to reduce the latency and improve the energy efficiency, few works have investigated the impact of limited batteries at the devices on the FEEL. Motivated by this, we study the battery-constrained FEEL, where the UAVs can adjust their operating CPU-frequency to prolong the battery life and avoid withdrawing from federated learning training untimely. We optimize the system by jointly allocating the computational resource and wireless bandwidth in time-varying environments. To solve this optimization problem, we employ a deep deterministic policy gradient (DDPG) based strategy, where a linear combination of latency and energy consumption is used to evaluate the system cost. Simulation results are finally demonstrated to show that the proposed strategy outperforms the conventional ones. In particular, it enables all the devices to complete all rounds of FEEL with limited batteries and meanwhile reduce the system cost effectively.
A Comprehensive Survey on 6G Networks:Applications, Core Services, Enabling Technologies, and Future Challenges
Venue: arXiv cs.NI 2021
Year: 2021

Cellular Internet of Things (IoT) is considered as de facto paradigm to improve the communication and computation systems. Cellular IoT connects massive number of physical and virtual objects to the Internet using cellular networks. The latest generation of cellular networks, e.g. fifth-generation (5G), use evolutionary and revolutionary technologies to notably improve the performance of wireless networks. However, given the envisioned new use-cases, e.g., holographic communication, and the ever-increasing deployment of massive smart-physical end-devices in IoT, the volume of network traffic has considerably raised, and therefore, the current generation of mobile networks cannot wholly meet the ever-increasing demands. Hence, it is envisioned that the next generation, sixth generation (6G) networks, need to play a critical role to alleviate such challenges in IoT by providing new communication services, network capacity, and ultra-low latency communications (uRLLC). In this paper, first, the need for 6G networks is discussed. Then, the potential 6G requirements and trends, as well as the latest research activities related to 6G are introduced e.g., Tactile Internet and Terahertz (THz). Furthermore, the key performance indicators, applications, new services, and the potential key enabling technologies for 6G networks are presented. Finally, several potential unresolved challenges for future 6G networks are presented.
Beyond traditional assumptions in fair machine learning
Venue: arXiv cs.LG 2021
Year: 2021

This thesis scrutinizes common assumptions underlying traditional machine learning approaches to fairness in consequential decision making. After challenging the validity of these assumptions in real-world applications, we propose ways to move forward when they are violated. First, we show that group fairness criteria purely based on statistical properties of observed data are fundamentally limited. Revisiting this limitation from a causal viewpoint we develop a more versatile conceptual framework, causal fairness criteria, and first algorithms to achieve them. We also provide tools to analyze how sensitive a believed-to-be causally fair algorithm is to misspecifications of the causal graph. Second, we overcome the assumption that sensitive data is readily available in practice. To this end we devise protocols based on secure multi-party computation to train, validate, and contest fair decision algorithms without requiring users to disclose their sensitive data or decision makers to disclose their models. Finally, we also accommodate the fact that outcome labels are often only observed when a certain decision has been made. We suggest a paradigm shift away from training predictive models towards directly learning decisions to relax the traditional assumption that labels can always be recorded. The main contribution of this thesis is the development of theoretically substantiated and practically feasible methods to move research on fair machine learning closer to real-world applications.
Self-Supervised Pretraining for RGB-D Salient Object Detection
Venue: arXiv cs.CV 2021
Year: 2021

Existing CNNs-Based RGB-D salient object detection (SOD) networks are all required to be pretrained on the ImageNet to learn the hierarchy features which helps provide a good initialization. However, the collection and annotation of large-scale datasets are time-consuming and expensive. In this paper, we utilize self-supervised representation learning (SSL) to design two pretext tasks: the cross-modal auto-encoder and the depth-contour estimation. Our pretext tasks require only a few and unlabeled RGB-D datasets to perform pretraining, which makes the network capture rich semantic contexts and reduce the gap between two modalities, thereby providing an effective initialization for the downstream task. In addition, for the inherent problem of cross-modal fusion in RGB-D SOD, we propose a consistency-difference aggregation (CDA) module that splits a single feature fusion into multi-path fusion to achieve an adequate perception of consistent and differential information. The CDA module is general and suitable for cross-modal and cross-level feature fusion. Extensive experiments on six benchmark datasets show that our self-supervised pretrained model performs favorably against most state-of-the-art methods pretrained on ImageNet. The source code will be publicly available at \textcolor{red}{\url{https://github.com/Xiaoqi-Zhao-DLUT/SSLSOD}}.
Aquanims: Area-Preserving Animated Transitions in Statistical Data Graphics based on a Hydraulic Metaphor
Venue: arXiv cs.HC 2021
Year: 2021

We propose "aquanims" as new design metaphors for animated transitions that preserve displayed areas during the transformation. Animated transitions are used to facilitate understanding of graphical transformations between different visualizations. Area is key information to preserve during filtering or ordering transitions of area-based charts like bar charts, histograms, treemaps, or mosaic plots. As liquids are incompressible fluids, we use a hydraulic metaphor to convey the sense of area preservation during animated transitions: in aquanims, graphical objects can change shape, position, color, and even connectedness but not displayed area, as for a liquid contained in a transparent vessel or transferred between such vessels communicating through hidden pipes. We present various aquanims for product plots like bar charts and histograms to accommodate changes in data, in the ordering of bars or in a number of bins, and to provide animated tips. We also consider confusion matrices visualized as fluctuation diagrams and mosaic plots, and show how aquanims can be used to ease the understanding of different classification errors of real data.
A Model of WiFi Performance With Bounded Latency
Venue: arXiv cs.NI 2021
Year: 2021

In September 2020, the Broadband Forum published a new industry standard for measuring network quality. The standard centers on the notion of quality attenuation. Quality attenuation is a measure of the distribution of latency and packet loss between two points connected by a network path. A vital feature of the quality attenuation idea is that we can express detailed application requirements and network performance measurements in the same mathematical framework. Performance requirements and measurements are both modeled as latency distributions. To the best of our knowledge, existing models of the 802.11 WiFi protocol do not permit the calculation of complete latency distributions without assuming steady-state operation. We present a novel model of the WiFi protocol. Instead of computing throughput numbers from a steady-state analysis of a Markov chain, we explicitly model latency and packet loss. Explicitly modeling latency and loss allows for both transient and steady-state analysis of latency distributions, and we can derive throughput numbers from the latency results. Our model is, therefore, more general than the standard Markov chain methods. We reproduce several known results with this method. Using transient analysis, we derive bounds on WiFi throughput under the requirement that latency and packet loss must be bounded.
Optimal strategies for reject option classifiers
Venue: arXiv cs.LG 2021
Year: 2021

In classification with a reject option, the classifier is allowed in uncertain cases to abstain from prediction. The classical cost-based model of a reject option classifier requires the cost of rejection to be defined explicitly. An alternative bounded-improvement model, avoiding the notion of the reject cost, seeks for a classifier with a guaranteed selective risk and maximal cover. We coin a symmetric definition, the bounded-coverage model, which seeks for a classifier with minimal selective risk and guaranteed coverage. We prove that despite their different formulations the three rejection models lead to the same prediction strategy: a Bayes classifier endowed with a randomized Bayes selection function. We define a notion of a proper uncertainty score as a scalar summary of prediction uncertainty sufficient to construct the randomized Bayes selection function. We propose two algorithms to learn the proper uncertainty score from examples for an arbitrary black-box classifier. We prove that both algorithms provide Fisher consistent estimates of the proper uncertainty score and we demonstrate their efficiency on different prediction problems including classification, ordinal regression and structured output classification.
Interleaving Graph Search and Trajectory Optimization for Aggressive Quadrotor Flight
Venue: arXiv cs.RO 2021
Year: 2021

Quadrotors can achieve aggressive flight by tracking complex maneuvers and rapidly changing directions. Planning for aggressive flight with trajectory optimization could be incredibly fast, even in higher dimensions, and can account for dynamics of the quadrotor, however, only provides a locally optimal solution. On the other hand, planning with discrete graph search can handle non-convex spaces to guarantee optimality but suffers from exponential complexity with the dimension of search. We introduce a framework for aggressive quadrotor trajectory generation with global reasoning capabilities that combines the best of trajectory optimization and discrete graph search. Specifically, we develop a novel algorithmic framework that interleaves these two methods to complement each other and generate trajectories with provable guarantees on completeness up to discretization. We demonstrate and quantitatively analyze the performance of our algorithm in challenging simulation environments with narrow gaps that create severe attitude constraints and push the dynamic capabilities of the quadrotor. Experiments show the benefits of the proposed algorithmic framework over standalone trajectory optimization and graph search-based planning techniques for aggressive quadrotor flight.
A Symmetric Lambda-Calculus Corresponding to the Negation-Free Bilateral Natural Deduction
Venue: arXiv cs.LO 2021
Year: 2021

Filinski constructed a symmetric lambda-calculus consisting of expressions and continuations which are symmetric, and functions which have duality. In his calculus, functions can be encoded to expressions and continuations using primitive operators. That is, the duality of functions is not derived in the calculus but adopted as a principle of the calculus. In this paper, we propose a simple symmetric lambda-calculus corresponding to the negation-free natural deduction based bilateralism in proof-theoretic semantics. In our calculus, continuation types are represented as not negations of formulae but formulae with negative polarity. Function types are represented as the implication and but-not connectives in intuitionistic and paraconsistent logics, respectively. Our calculus is not only simple but also powerful as it includes a call-value calculus corresponding to the call-by-value dual calculus invented by Wadler. We show that mutual transformations between expressions and continuations are definable in our calculus to justify the duality of functions. We also show that every typable function has dual types. Thus, the duality of function is derived from bilateralism.
DigitalExposome: Quantifying the Urban Environment Influence on Wellbeing based on Real-Time Multi-Sensor Fusion and Deep Belief Network
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we define the term 'DigitalExposome' as a conceptual framework that takes us closer towards understanding the relationship between environment, personal characteristics, behaviour and wellbeing using multimodel mobile sensing technology. Specifically, we simultaneously collected (for the first time) multi-sensor data including urban environmental factors (e.g. air pollution including: PM1, PM2.5, PM10, Oxidised, Reduced, NH3 and Noise, People Count in the vicinity), body reaction (physiological reactions including: EDA, HR, HRV, Body Temperature, BVP and movement) and individuals' perceived responses (e.g. self-reported valence) in urban settings. Our users followed a pre-specified urban path and collected the data using a comprehensive sensing edge devices. The data is instantly fused, time-stamped and geo-tagged at the point of collection. A range of multivariate statistical analysis techniques have been applied including Principle Component Analysis, Regression and spatial visualisations to unravel the relationship between the variables. Results showed that EDA and Heart Rate Variability HRV are noticeably impacted by the level of Particulate Matters (PM) in the environment well with the environmental variables. Furthermore, we adopted Deep Belief Network to extract features from the multimodel data feed which outperformed Convolutional Neural Network and achieved up to (a=80.8%, σ=0.001) accuracy.
RecSSD: Near Data Processing for Solid State Drive Based Recommendation Inference
Venue: arXiv cs.AR 2021
Year: 2021

Neural personalized recommendation models are used across a wide variety of datacenter applications including search, social media, and entertainment. State-of-the-art models comprise large embedding tables that have billions of parameters requiring large memory capacities. Unfortunately, large and fast DRAM-based memories levy high infrastructure costs. Conventional SSD-based storage solutions offer an order of magnitude larger capacity, but have worse read latency and bandwidth, degrading inference performance. RecSSD is a near data processing based SSD memory system customized for neural recommendation inference that reduces end-to-end model inference latency by 2X compared to using COTS SSDs across eight industry-representative models.
A linearized framework and a new benchmark for model selection for fine-tuning
Venue: arXiv cs.CV 2021
Year: 2021

Fine-tuning from a collection of models pre-trained on different domains (a "model zoo") is emerging as a technique to improve test accuracy in the low-data regime. However, model selection, i.e. how to pre-select the right model to fine-tune from a model zoo without performing any training, remains an open topic. We use a linearized framework to approximate fine-tuning, and introduce two new baselines for model selection -- Label-Gradient and Label-Feature Correlation. Since all model selection algorithms in the literature have been tested on different use-cases and never compared directly, we introduce a new comprehensive benchmark for model selection comprising of: i) A model zoo of single and multi-domain models, and ii) Many target tasks. Our benchmark highlights accuracy gain with model zoo compared to fine-tuning Imagenet models. We show our model selection baseline can select optimal models to fine-tune in few selections and has the highest ranking correlation to fine-tuning accuracy compared to existing algorithms.
Relaxed Clustered Hawkes Process for Procrastination Modeling in MOOCs
Venue: arXiv cs.LG 2021
Year: 2021

Hawkes processes have been shown to be efficient in modeling bursty sequences in a variety of applications, such as finance and social network activity analysis. Traditionally, these models parameterize each process independently and assume that the history of each point process can be fully observed. Such models could however be inefficient or even prohibited in certain real-world applications, such as in the field of education, where such assumptions are violated. Motivated by the problem of detecting and predicting student procrastination in students Massive Open Online Courses (MOOCs) with missing and partially observed data, in this work, we propose a novel personalized Hawkes process model (RCHawkes-Gamma) that discovers meaningful student behavior clusters by jointly learning all partially observed processes simultaneously, without relying on auxiliary features. Our experiments on both synthetic and real-world education datasets show that RCHawkes-Gamma can effectively recover student clusters and their temporal procrastination dynamics, resulting in better predictive performance of future student activities. Our further analyses of the learned parameters and their association with student delays show that the discovered student clusters unveil meaningful representations of various procrastination behaviors in students.
Certified evaluations of Hölder continuous functions at roots of polynomials
Venue: arXiv cs.SC 2021
Year: 2021

Various methods can obtain certified estimates for roots of polynomials. Many applications in science and engineering additionally utilize the value of functions evaluated at roots. For example, critical values are obtained by evaluating an objective function at critical points. For analytic evaluation functions, Newton's method naturally applies to yield certified estimates. These estimates no longer apply, however, for Hölder continuous functions, which are a generalization of Lipschitz continuous functions where continuous derivatives need not exist. This work develops and analyzes an alternative approach for certified estimates of evaluating locally Hölder continuous functions at roots of polynomials. An implementation of the method in Maple demonstrates efficacy and efficiency.
Partial Non-Orthogonal Multiple Access (NOMA) in Downlink Poisson Networks
Venue: arXiv cs.IT 2021
Year: 2021

Non-orthogonal multiple access (NOMA) allows users sharing a resource-block to efficiently reuse spectrum and improve cell sum rate $\mathcal{R}_{\rm tot}$ at the expense of increased interference. Orthogonal multiple access (OMA), on the other hand, guarantees higher coverage. We introduce partial-NOMA in a large two-user downlink network to provide both throughput and reliability. The associated partial overlap controls interference while still offering spectrum reuse. The nature of the partial overlap also allows us to employ receive-filtering to further suppress interference. For signal decoding in our partial-NOMA setup, we propose a new technique called flexible successive interference cancellation (FSIC) decoding. We plot the rate region abstraction and compare with OMA and NOMA. We formulate a problem to maximize $\mathcal{R}_{\rm tot}$ constrained to a minimum throughput requirement for each user and propose an algorithm to find a feasible resource allocation efficiently. Our results show that partial-NOMA allows greater flexibility in terms of performance. Partial-NOMA can also serve users that NOMA cannot. We also show that with appropriate parameter selection and resource allocation, partial-NOMA can outperform NOMA.
Actor-Critic Learning Based QoS-Aware Scheduler for Reconfigurable Wireless Networks
Venue: arXiv cs.NI 2021
Year: 2021

The flexibility offered by reconfigurable wireless networks, provide new opportunities for various applications such as online AR/VR gaming, high-quality video streaming and autonomous vehicles, that desire high-bandwidth, reliable and low-latency communications. These applications come with very stringent Quality of Service (QoS) requirements and increase the burden over mobile networks. Currently, there is a huge spectrum scarcity due to the massive data explosion and this problem can be solved by helps of Reconfigurable Wireless Networks (RWNs) where nodes have reconfiguration and perception capabilities. Therefore, a necessity of AI-assisted algorithms for resource block allocation is observed. To tackle this challenge, in this paper, we propose an actor-critic learning-based scheduler for allocating resource blocks in a RWN. Various traffic types with different QoS levels are assigned to our agents to provide more realistic results. We also include mobility in our simulations to increase the dynamicity of networks. The proposed model is compared with another actor-critic model and with other traditional schedulers; proportional fair (PF) and Channel and QoS Aware (CQA) techniques. The proposed models are evaluated by considering the delay experienced by user equipment (UEs), successful transmissions and head-of-the-line delays. The results show that the proposed model noticeably outperforms other techniques in different aspects.
DRIV100: In-The-Wild Multi-Domain Dataset and Evaluation for Real-World Domain Adaptation of Semantic Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Together with the recent advances in semantic segmentation, many domain adaptation methods have been proposed to overcome the domain gap between training and deployment environments. However, most previous studies use limited combinations of source/target datasets, and domain adaptation techniques have never been thoroughly evaluated in a more challenging and diverse set of target domains. This work presents a new multi-domain dataset DRIV100 for benchmarking domain adaptation techniques on in-the-wild road-scene videos collected from the Internet. The dataset consists of pixel-level annotations for 100 videos selected to cover diverse scenes/domains based on two criteria; human subjective judgment and an anomaly score judged using an existing road-scene dataset. We provide multiple manually labeled ground-truth frames for each video, enabling a thorough evaluation of video-level domain adaptation where each video independently serves as the target domain. Using the dataset, we quantify domain adaptation performances of state-of-the-art methods and clarify the potential and novel challenges of domain adaptation techniques. The dataset is available at https://doi.org/10.5281/zenodo.4389243.
Time Series (re)sampling using Generative Adversarial Networks
Venue: arXiv cs.LG 2021
Year: 2021

We propose a novel bootstrap procedure for dependent data based on Generative Adversarial networks (GANs). We show that the dynamics of common stationary time series processes can be learned by GANs and demonstrate that GANs trained on a single sample path can be used to generate additional samples from the process. We find that temporal convolutional neural networks provide a suitable design for the generator and discriminator, and that convincing samples can be generated on the basis of a vector of iid normal noise. We demonstrate the finite sample properties of GAN sampling and the suggested bootstrap using simulations where we compare the performance to circular block bootstrapping in the case of resampling an AR(1) time series processes. We find that resampling using the GAN can outperform circular block bootstrapping in terms of empirical coverage.