OpenRaiser/Intern-Atlas huggingface dataset:
A General Framework for Sample-Efficient Function Approximation in Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

With the increasing need for handling large state and action spaces, general function approximation has become a key technique in reinforcement learning (RL). In this paper, we propose a general framework that unifies model-based and model-free RL, and an Admissible Bellman Characterization (ABC) class that subsumes nearly all Markov Decision Process (MDP) models in the literature for tractable RL. We propose a novel estimation function with decomposable structural properties for optimization-based exploration and the functional eluder dimension as a complexity measure of the ABC class. Under our framework, a new sample-efficient algorithm namely OPtimization-based ExploRation with Approximation (OPERA) is proposed, achieving regret bounds that match or improve over the best-known results for a variety of MDP models. In particular, for MDPs with low Witness rank, under a slightly stronger assumption, OPERA improves the state-of-the-art sample complexity results by a factor of $dH$. Our framework provides a generic interface to design and analyze new RL models and algorithms.
FedTrees: A Novel Computation-Communication Efficient Federated Learning Framework Investigated in Smart Grids
Venue: arXiv cs.LG 2022
Year: 2022

Smart energy performance monitoring and optimisation at the supplier and consumer levels is essential to realising smart cities. In order to implement a more sustainable energy management plan, it is crucial to conduct a better energy forecast. The next-generation smart meters can also be used to measure, record, and report energy consumption data, which can be used to train machine learning (ML) models for predicting energy needs. However, sharing fine-grained energy data and performing centralised learning may compromise users' privacy and leave them vulnerable to several attacks. This study addresses this issue by utilising federated learning (FL), an emerging technique that performs ML model training at the user level, where data resides. We introduce FedTrees, a new, lightweight FL framework that benefits from the outstanding features of ensemble learning. Furthermore, we developed a delta-based early stopping algorithm to monitor FL training and stop it when it does not need to continue. The simulation results demonstrate that FedTrees outperforms the most popular federated averaging (FedAvg) framework and the baseline Persistence model for providing accurate energy forecasting patterns while taking only 2% of the computation time and 13% of the communication rounds compared to FedAvg, saving considerable amounts of computation and communication resources.
CEREAL: Few-Sample Clustering Evaluation
Venue: arXiv cs.LG 2022
Year: 2022

Evaluating clustering quality with reliable evaluation metrics like normalized mutual information (NMI) requires labeled data that can be expensive to annotate. We focus on the underexplored problem of estimating clustering quality with limited labels. We adapt existing approaches from the few-sample model evaluation literature to actively sub-sample, with a learned surrogate model, the most informative data points for annotation to estimate the evaluation metric. However, we find that their estimation can be biased and only relies on the labeled data. To that end, we introduce CEREAL, a comprehensive framework for few-sample clustering evaluation that extends active sampling approaches in three key ways. First, we propose novel NMI-based acquisition functions that account for the distinctive properties of clustering and uncertainties from a learned surrogate model. Next, we use ideas from semi-supervised learning and train the surrogate model with both the labeled and unlabeled data. Finally, we pseudo-label the unlabeled data with the surrogate model. We run experiments to estimate NMI in an active sampling pipeline on three datasets across vision and language. Our results show that CEREAL reduces the area under the absolute error curve by up to 57% compared to the best sampling baseline. We perform an extensive ablation study to show that our framework is agnostic to the choice of clustering algorithm and evaluation metric. We also extend CEREAL from clusterwise annotations to pairwise annotations. Overall, CEREAL can efficiently evaluate clustering with limited human annotations.
Shockwave: Fair and Efficient Cluster Scheduling for Dynamic Adaptation in Machine Learning
Venue: arXiv cs.DC 2022
Year: 2022

Dynamic adaptation has become an essential technique in accelerating distributed machine learning (ML) training. Recent studies have shown that dynamically adjusting model structure (e.g., lottery ticket hypothesis) or hyperparameters (e.g., batch size) can significantly accelerate training without sacrificing accuracy. However, existing ML cluster schedulers are not designed to handle dynamic adaptation. We show that existing schemes fail to provide fairness and degrade system efficiency when the training throughput changes over time under dynamic adaptation. We design Shockwave, a scheduler with future planning that builds on two key ideas. First, Shockwave extends classic market theory from static settings to dynamic settings to co-optimize efficiency and fairness. Second, Shockwave utilizes stochastic dynamic programming to handle dynamic changes. We build a system for Shockwave and validate its performance with both trace-driven simulation and cluster experiments. Results show that for traces of ML jobs with dynamic adaptation, Shockwave improves makespan by 1.3X and fairness by 2X when compared with existing fair scheduling schemes.
Coalitional Game-Theoretical Approach to Coinvestment with Application to Edge Computing
Venue: arXiv cs.GT 2022
Year: 2022

We propose in this paper a coinvestment plan between several stakeholders of different types, namely a physical network owner, operating network nodes, e.g. a network operator or a tower company, and a set of service providers willing to use these resources to provide services as video streaming, augmented reality, autonomous driving assistance, etc. One such scenario is that of deployment of Edge Computing resources. Indeed, although the latter technology is ready, the high Capital Expenditure (CAPEX) cost of such resources is the barrier to its deployment. For this reason, a solid economical framework to guide the investment and the returns of the stakeholders is key to solve this issue. We formalize the coinvestment framework using coalitional game theory. We provide a solution to calculate how to divide the profits and costs among the stakeholders, taking into account their characteristics: traffic load, revenues, utility function. We prove that it is always possible to form the grand coalition composed of all the stakeholders, by showing that our game is convex. We derive the payoff of the stakeholders using the Shapley value concept, and elaborate on some properties of our game. We show our solution in simulation.
FAST-LIO, Then Bayesian ICP, Then GTSFM
Venue: arXiv cs.RO 2022
Year: 2022

For the Hilti Challenge 2022, we created two systems, one building upon the other. The first system is FL2BIPS which utilizes the iEKF algorithm FAST-LIO2 and Bayesian ICP PoseSLAM, whereas the second system is GTSFM, a structure from motion pipeline with factor graph backend optimization powered by GTSAM
EAPruning: Evolutionary Pruning for Vision Transformers and CNNs
Venue: arXiv cs.CV 2022
Year: 2022

Structured pruning greatly eases the deployment of large neural networks in resource-constrained environments. However, current methods either involve strong domain expertise, require extra hyperparameter tuning, or are restricted only to a specific type of network, which prevents pervasive industrial applications. In this paper, we undertake a simple and effective approach that can be easily applied to both vision transformers and convolutional neural networks. Specifically, we consider pruning as an evolution process of sub-network structures that inherit weights through reconstruction techniques. We achieve a 50% FLOPS reduction for ResNet50 and MobileNetV1, leading to 1.37x and 1.34x speedup respectively. For DeiT-Base, we reach nearly 40% FLOPs reduction and 1.4x speedup. Our code will be made available.
Configuration Tracking Control of a Multi-Segment Soft Robotic Arm Using a Cosserat Rod Model
Venue: arXiv cs.RO 2022
Year: 2022

Controlling soft continuum robotic arms is challenging due to their hyper-redundancy and dexterity. In this paper we demonstrate, for the first time, closed-loop control of the configuration space variables of a soft robotic arm, composed of independently controllable segments, using a Cosserat rod model of the robot and the distributed sensing and actuation capabilities of the segments. Our controller solves the inverse dynamic problem by simulating the Cosserat rod model in MATLAB using a computationally efficient numerical solution scheme, and it applies the computed control output to the actual robot in real time. The position and orientation of the tip of each segment are measured in real time, while the remaining unknown variables that are needed to solve the inverse dynamics are estimated simultaneously in the simulation. We implement the controller on a multi-segment silicone robotic arm with pneumatic actuation, using a motion capture system to measure the segments' positions and orientations. The controller is used to reshape the arm into configurations that are achieved through different combinations of bending and extension deformations in 3D space. The resulting tracking performance indicates the effectiveness of the controller and the accuracy of the simulated Cosserat rod model that is used to estimate the unmeasured variables.
Structure-Aware NeRF without Posed Camera via Epipolar Constraint
Venue: arXiv cs.CV 2022
Year: 2022

The neural radiance field (NeRF) for realistic novel view synthesis requires camera poses to be pre-acquired by a structure-from-motion (SfM) approach. This two-stage strategy is not convenient to use and degrades the performance because the error in the pose extraction can propagate to the view synthesis. We integrate the pose extraction and view synthesis into a single end-to-end procedure so they can benefit from each other. For training NeRF models, only RGB images are given, without pre-known camera poses. The camera poses are obtained by the epipolar constraint in which the identical feature in different views has the same world coordinates transformed from the local camera coordinates according to the extracted poses. The epipolar constraint is jointly optimized with pixel color constraint. The poses are represented by a CNN-based deep network, whose input is the related frames. This joint optimization enables NeRF to be aware of the scene's structure that has an improved generalization performance. Extensive experiments on a variety of scenes demonstrate the effectiveness of the proposed approach. Code is available at https://github.com/XTU-PR-LAB/SaNerf.
Integrating Conventional Headway Control with Reinforcement Learning to Avoid Bus Bunching
Venue: arXiv cs.MA 2022
Year: 2022

Bus bunching is a natural-occurring phenomenon that undermines the efficiency and stability of the public transportation system. The mainstream solutions control the bus to intentionally stay longer at certain stations. Existing control methods include conventional methods that provide a formula to calculate the control time and reinforcement learning (RL) methods that determine the control policy through repeated interactions with the system. In this paper, we propose an integrated proximal policy optimization model with dual-headway (IPPO-DH). IPPO-DH integrates the conventional headway control with reinforcement learning, so that it acquires the advantages of both algorithms -- it is more efficient in normal environments and more stable in harsh ones. To demonstrate such an advantage, we design a bus simulation environment and compare IPPO-DH with RL and several conventional methods. The results show that the proposed model maintains the application value of the conventional method by avoiding the instability of the RL method in certain environments, and improves the efficiency compared with the conventional control, shedding new light on real-world bus transit system optimization.
Digital Library Initiatives in North East India: A Survey
Venue: arXiv cs.DL 2022
Year: 2022

This is a survey of digital library initiative of North East India. The recent initiative by the government of India towards the digitization is reflected in various digitation programs. The secondary sources of data are used to map the 16 digital library initiatives in eight north east state of India. The study has observed that digital library in true sense is perhaps lacking. Many of the digital libraries are not accessible from the outside and lack regular maintenance. In this context, a national level policy initiative is the need of the hour including various stakeholders from the academics, library professionals etc. The study also comes up with various important observations and policy suggestions which may be helpful for scholar, librarians, policy and decision makers in the government.
An experimental study of algorithms for obtaining a singly connected subgraph
Venue: arXiv cs.DS 2022
Year: 2022

A directed graph G = (V,E) is singly connected if for any two vertices v, u of V, the directed graph G contains at most one simple path from v to u. In this paper, we study different algorithms to find a feasible but necessarily optimal solution to the following problem. Given a directed acyclic graph G = (V, E), find a subset H of E of minimum size such that the subgraph (V, E-H) is singly connected. Moreover, we prove that this problem can be solved in polynomial time for a special kind of directed graphs.
Solar Power Time Series Forecasting Utilising Wavelet Coefficients
Venue: arXiv cs.LG 2022
Year: 2022

Accurate and reliable prediction of Photovoltaic (PV) power output is critical to electricity grid stability and power dispatching capabilities. However, Photovoltaic (PV) power generation is highly volatile and unstable due to different reasons. The Wavelet Transform (WT) has been utilised in time series applications, such as Photovoltaic (PV) power prediction, to model the stochastic volatility and reduce prediction errors. Yet the existing Wavelet Transform (WT) approach has a limitation in terms of time complexity. It requires reconstructing the decomposed components and modelling them separately and thus needs more time for reconstruction, model configuration and training. The aim of this study is to improve the efficiency of applying Wavelet Transform (WT) by proposing a new method that uses a single simplified model. Given a time series and its Wavelet Transform (WT) coefficients, it trains one model with the coefficients as features and the original time series as labels. This eliminates the need for component reconstruction and training numerous models. This work contributes to the day-ahead aggregated solar Photovoltaic (PV) power time series prediction problem by proposing and comprehensively evaluating a new approach of employing WT. The proposed approach is evaluated using 17 months of aggregated solar Photovoltaic (PV) power data from two real-world datasets. The evaluation includes the use of a variety of prediction models, including Linear Regression, Random Forest, Support Vector Regression, and Convolutional Neural Networks. The results indicate that using a coefficients-based strategy can give predictions that are comparable to those obtained using the components-based approach while requiring fewer models and less computational time.
ML for Location Prediction Using RSSI On WiFi 2.4 GHZ Frequency Band
Venue: arXiv cs.CR 2022
Year: 2022

For decades, the determination of an objects location has been implemented utilizing different technologies. Despite GPS (Global Positioning System) provides a scalable efficient and cost effective location services however the satellite emitted signals cannot be exploited indoor to effectively determine the location. In contrast to GPS which is a cost effective localization technology for outdoor locations several technologies have been studied for indoor localization. These include Wireless Fidelity (Wi-Fi) Bluetooth Low Energy (BLE) and Received Signal Strength Indicator (RSSI) etc. This paper presents an enhanced method of using RSSI as a mean to determine an objects location by applying some Machine Learning (ML) concepts. The binary classification is defined by considering the adjacency of the coordinates denoting objects locations. The proposed features were tested empirically via multiple classifiers that achieved a maximum of 96 percent accuracy.
NeuroEvo: A Cloud-based Platform for Automated Design and Training of Neural Networks using Evolutionary and Particle Swarm Algorithms
Venue: arXiv cs.NE 2022
Year: 2022

Evolutionary algorithms (EAs) provide unique advantages for optimizing neural networks in complex search spaces. This paper introduces a new web platform, NeuroEvo (neuroevo.io), that allows users to interactively design and train neural network classifiers using evolutionary and particle swarm algorithms. The classification problem and training data are provided by the user and, upon completion of the training process, the best classifier is made available to download and implement in Python, Java, and JavaScript. NeuroEvo is a cloud-based application that leverages GPU parallelization to improve the speed with which the independent evolutionary steps, such as mutation, crossover, and fitness evaluation, are executed across the population. This paper outlines the training algorithms and opportunities for users to specify design decisions and hyperparameter settings. The algorithms described in this paper are also made available as a Python package, neuroevo (PyPI: https://pypi.org/project/neuroevo/).
DeltaBound Attack: Efficient decision-based attack in low queries regime
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks and other machine learning systems, despite being extremely powerful and able to make predictions with high accuracy, are vulnerable to adversarial attacks. We proposed the DeltaBound attack: a novel, powerful attack in the hard-label setting with $\ell_2$ norm bounded perturbations. In this scenario, the attacker has only access to the top-1 predicted label of the model and can be therefore applied to real-world settings such as remote API. This is a complex problem since the attacker has very little information about the model. Consequently, most of the other techniques present in the literature require a massive amount of queries for attacking a single example. Oppositely, this work mainly focuses on the evaluation of attack's power in the low queries regime $\leq 1000$ queries) with $\ell_2$ norm in the hard-label settings. We find that the DeltaBound attack performs as well and sometimes better than current state-of-the-art attacks while remaining competitive across different kinds of models. Moreover, we evaluate our method against not only deep neural networks, but also non-deep learning models, such as Gradient Boosting Decision Trees and Multinomial Naive Bayes.
PathFinder: Discovering Decision Pathways in Deep Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Explainability is becoming an increasingly important topic for deep neural networks. Though the operation in convolutional layers is easier to understand, processing becomes opaque in fully-connected layers. The basic idea in our work is that each instance, as it flows through the layers, causes a different activation pattern in the hidden layers and in our Paths methodology, we cluster these activation vectors for each hidden layer and then see how the clusters in successive layers connect to one another as activation flows from the input layer to the output. We find that instances of the same class follow a small number of cluster sequences over the layers, which we name ``decision paths." Such paths explain how classification decisions are typically made, and also help us determine outliers that follow unusual paths. We also propose using the Sankey diagram to visualize such pathways. We validate our method with experiments on two feed-forward networks trained on MNIST and CELEB data sets, and one recurrent network trained on PenDigits.
CGELBank: CGEL as a Framework for English Syntax Annotation
Venue: arXiv cs.CL 2022
Year: 2022

We introduce the syntactic formalism of the \textit{Cambridge Grammar of the English Language} (CGEL) to the world of treebanking through the CGELBank project. We discuss some issues in linguistic analysis that arose in adapting the formalism to corpus annotation, followed by quantitative and qualitative comparisons with parallel UD and PTB treebanks. We argue that CGEL provides a good tradeoff between comprehensiveness of analysis and usability for annotation, which motivates expanding the treebank with automatic conversion in the future.
Understanding Adversarial Robustness Against On-manifold Adversarial Examples
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks (DNNs) are shown to be vulnerable to adversarial examples. A well-trained model can be easily attacked by adding small perturbations to the original data. One of the hypotheses of the existence of the adversarial examples is the off-manifold assumption: adversarial examples lie off the data manifold. However, recent research showed that on-manifold adversarial examples also exist. In this paper, we revisit the off-manifold assumption and want to study a question: at what level is the poor performance of neural networks against adversarial attacks due to on-manifold adversarial examples? Since the true data manifold is unknown in practice, we consider two approximated on-manifold adversarial examples on both real and synthesis datasets. On real datasets, we show that on-manifold adversarial examples have greater attack rates than off-manifold adversarial examples on both standard-trained and adversarially-trained models. On synthetic datasets, theoretically, We prove that on-manifold adversarial examples are powerful, yet adversarial training focuses on off-manifold directions and ignores the on-manifold adversarial examples. Furthermore, we provide analysis to show that the properties derived theoretically can also be observed in practice. Our analysis suggests that on-manifold adversarial examples are important, and we should pay more attention to on-manifold adversarial examples for training robust models.
Improved Stein Variational Gradient Descent with Importance Weights
Venue: arXiv cs.LG 2022
Year: 2022

Stein Variational Gradient Descent (SVGD) is a popular sampling algorithm used in various machine learning tasks. It is well known that SVGD arises from a discretization of the kernelized gradient flow of the Kullback-Leibler divergence $D_{KL}\left(\cdot\midπ\right)$, where $π$ is the target distribution. In this work, we propose to enhance SVGD via the introduction of importance weights, which leads to a new method for which we coin the name $β$-SVGD. In the continuous time and infinite particles regime, the time for this flow to converge to the equilibrium distribution $π$, quantified by the Stein Fisher information, depends on $ρ_0$ and $π$ very weakly. This is very different from the kernelized gradient flow of Kullback-Leibler divergence, whose time complexity depends on $D_{KL}\left(ρ_0\midπ\right)$. Under certain assumptions, we provide a descent lemma for the population limit $β$-SVGD, which covers the descent lemma for the population limit SVGD when $β\to 0$. We also illustrate the advantages of $β$-SVGD over SVGD by experiments.