OpenRaiser/Intern-Atlas huggingface dataset:
Instance Correction for Learning with Open-set Noisy Labels
Venue: arXiv cs.LG 2021
Year: 2021

The problem of open-set noisy labels denotes that part of training data have a different label space that does not contain the true class. Lots of approaches, e.g., loss correction and label correction, cannot handle such open-set noisy labels well, since they need training data and test data to share the same label space, which does not hold for learning with open-set noisy labels. The state-of-the-art methods thus employ the sample selection approach to handle open-set noisy labels, which tries to select clean data from noisy data for network parameters updates. The discarded data are seen to be mislabeled and do not participate in training. Such an approach is intuitive and reasonable at first glance. However, a natural question could be raised "can such data only be discarded during training?". In this paper, we show that the answer is no. Specifically, we discuss that the instances of discarded data could consist of some meaningful information for generalization. For this reason, we do not abandon such data, but use instance correction to modify the instances of the discarded data, which makes the predictions for the discarded data consistent with given labels. Instance correction are performed by targeted adversarial attacks. The corrected data are then exploited for training to help generalization. In addition to the analytical results, a series of empirical evidences are provided to justify our claims.
Towards Efficient Compressive Data Collection in the Internet of Things
Venue: arXiv cs.IT 2021
Year: 2021

It is of paramount importance to achieve efficient data collection in the Internet of Things (IoT). Due to the inherent structural properties (e.g., sparsity) existing in many signals of interest, compressive sensing (CS) technology has been extensively used for data collection in IoT to improve both accuracy and energy efficiency. Apart from the existing works which leverage CS as a channel coding scheme to deal with data loss during transmission, some recent results have started to employ CS as a source coding strategy. The frequently used projection matrices in these CS-based source coding schemes include dense random matrices (e.g., Gaussian matrices or Bernoulli matrices) and structured matrices (e.g., Toeplitz matrices). However, these matrices are either difficult to be implemented on resource-constrained IoT sensor nodes or have limited applicability. To address these issues, in this paper, we design a novel simple and efficient projection matrix, named sparse Gaussian matrix, which is easy and resource-saving to be implemented in practical IoT applications. We conduct both theoretical analysis and experimental evaluation of the designed sparse Gaussian matrix. The results demonstrate that employing the designed projection matrix to perform CS-based source coding could significantly save time and memory cost while ensuring satisfactory signal recovery performance.
Predicting Vehicles Trajectories in Urban Scenarios with Transformer Networks and Augmented Information
Venue: arXiv cs.CV 2021
Year: 2021

Understanding the behavior of road users is of vital importance for the development of trajectory prediction systems. In this context, the latest advances have focused on recurrent structures, establishing the social interaction between the agents involved in the scene. More recently, simpler structures have also been introduced for predicting pedestrian trajectories, based on Transformer Networks, and using positional information. They allow the individual modelling of each agent's trajectory separately without any complex interaction terms. Our model exploits these simple structures by adding augmented data (position and heading), and adapting their use to the problem of vehicle trajectory prediction in urban scenarios in prediction horizons up to 5 seconds. In addition, a cross-performance analysis is performed between different types of scenarios, including highways, intersections and roundabouts, using recent datasets (inD, rounD, highD and INTERACTION). Our model achieves state-of-the-art results and proves to be flexible and adaptable to different types of urban contexts.
Wireless Federated Learning with Limited Communication and Differential Privacy
Venue: arXiv cs.IT 2021
Year: 2021

This paper investigates the role of dimensionality reduction in efficient communication and differential privacy (DP) of the local datasets at the remote users for over-the-air computation (AirComp)-based federated learning (FL) model. More precisely, we consider the FL setting in which clients are prompted to train a machine learning model by simultaneous channel-aware and limited communications with a parameter server (PS) over a Gaussian multiple-access channel (GMAC), so that transmissions sum coherently at the PS globally aware of the channel coefficients. For this setting, an algorithm is proposed based on applying federated stochastic gradient descent (FedSGD) for training the minimum of a given loss function based on the local gradients, Johnson-Lindenstrauss (JL) random projection for reducing the dimension of the local updates, and artificial noise to further aid user's privacy. For this scheme, our results show that the local DP performance is mainly improved due to injecting noise of greater variance on each dimension while keeping the sensitivity of the projected vectors unchanged. This is while the convergence rate is slowed down compared to the case without dimensionality reduction. As the performance outweighs for the slower convergence, the trade-off between privacy and convergence is higher but is shown to lessen in high-dimensional regime yielding almost the same trade-off with much less communication cost.
Robust and accurate fine-grain power models for embedded systems with no on-chip PMU
Venue: arXiv cs.DC 2021
Year: 2021

This paper presents a novel approach to event-based power modelling for embedded platforms that do not have a Performance Monitoring Unit (PMU). The method involves complementing the target hardware platform, where the physical power data is measured, with another platform on which the CPU performance data, that is needed for model generation, can be collected. The methodology is used to generate accurate fine-grain power models for the the Gaisler GR712RC dual-core LEON3 fault-tolerant SPARC processor with on-board power sensors and no PMU. A Kintex UltraScale FPGA is used as the support platform to obtain the required CPU performance data, by running a soft-core representation of the dual-core LEON3 as on the GR712RC but with a PMU implementation. Both platforms execute the same benchmark set and data collection is synchronised using per-sample timestamps so that the power sensor data from the GR712RC board can be matched to the PMU data from the FPGA. The synchronised samples are then processed by the Robust Energy and Power Predictor Selection (REPPS) software in order to generate power models. The models achieve less than 2% power estimation error when validated on an industrial use-case and can successfully follow program phases, which makes them suitable for runtime power profiling.
Full-Resolution Encoder-Decoder Networks with Multi-Scale Feature Fusion for Human Pose Estimation
Venue: arXiv cs.CV 2021
Year: 2021

To achieve more accurate 2D human pose estimation, we extend the successful encoder-decoder network, simple baseline network (SBN), in three ways. To reduce the quantization errors caused by the large output stride size, two more decoder modules are appended to the end of the simple baseline network to get full output resolution. Then, the global context blocks (GCBs) are added to the encoder and decoder modules to enhance them with global context features. Furthermore, we propose a novel spatial-attention-based multi-scale feature collection and distribution module (SA-MFCD) to fuse and distribute multi-scale features to boost the pose estimation. Experimental results on the MS COCO dataset indicate that our network can remarkably improve the accuracy of human pose estimation over SBN, our network using ResNet34 as the backbone network can even achieve the same accuracy as SBN with ResNet152, and our networks can achieve superior results with big backbone networks.
Optimal virtual PON slicing to support ultra-low latency mesh traffic pattern in MEC-based Cloud-RAN
Venue: arXiv cs.NI 2021
Year: 2021

As progressive densification of cells, deployment of Cloud-RAN and \ac{MEC} are coming into reality to support the ultra-low latency with high reliability in 5G and beyond, it generates mesh traffic pattern across fronthaul network. This led to evolution of PON architectural enhancements with virtualization in order to support such mesh traffic pattern. However, allocation of virtual PON slices dynamically over such mesh-PON based fronthaul transport is becoming a research challenge. In this paper, we provide a mixed analytical-iterative model to compute optimal virtual PON slice allocation, providing mesh access connectivity with ultra-low end-to-end latency in next-generation MEC-based Cloud-RAN. Our proposed method can compute optimal virtual PON slice allocation in timescales compatible with real-time or near real-time operations.
Exposing Previously Undetectable Faults in Deep Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

Existing methods for testing DNNs solve the oracle problem by constraining the raw features (e.g. image pixel values) to be within a small distance of a dataset example for which the desired DNN output is known. But this limits the kinds of faults these approaches are able to detect. In this paper, we introduce a novel DNN testing method that is able to find faults in DNNs that other methods cannot. The crux is that, by leveraging generative machine learning, we can generate fresh test inputs that vary in their high-level features (for images, these include object shape, location, texture, and colour). We demonstrate that our approach is capable of detecting deliberately injected faults as well as new faults in state-of-the-art DNNs, and that in both cases, existing methods are unable to find these faults.
Fair Clustering Using Antidote Data
Venue: arXiv cs.LG 2021
Year: 2021

Clustering algorithms are widely utilized for many modern data science applications. This motivates the need to make outputs of clustering algorithms fair. Traditionally, new fair algorithmic variants to clustering algorithms are developed for specific notions of fairness. However, depending on the application context, different definitions of fairness might need to be employed. As a result, new algorithms and analysis need to be proposed for each combination of clustering algorithm and fairness definition. Additionally, each new algorithm would need to be reimplemented for deployment in a real-world system. Hence, we propose an alternate approach to group-level fairness in center-based clustering inspired by research on data poisoning attacks. We seek to augment the original dataset with a small number of data points, called antidote data. When clustering is undertaken on this new dataset, the output is fair, for the chosen clustering algorithm and fairness definition. We formulate this as a general bi-level optimization problem which can accommodate any center-based clustering algorithms and fairness notions. We then categorize approaches for solving this bi-level optimization for two different problem settings. Extensive experiments on different clustering algorithms and fairness notions show that our algorithms can achieve desired levels of fairness on many real-world datasets with a very small percentage of antidote data added. We also find that our algorithms achieve lower fairness costs and competitive clustering performance compared to other state-of-the-art fair clustering algorithms.
Porous Invariants
Venue: arXiv cs.LO 2021
Year: 2021

We introduce the notion of porous invariants for multipath (or branching/nondeterministic) affine loops over the integers; these invariants are not necessarily convex, and can in fact contain infinitely many 'holes'. Nevertheless, we show that in many cases such invariants can be automatically synthesised, and moreover can be used to settle (non-)reachability questions for various interesting classes of affine loops and target sets.
Public Goods Games in Directed Networks
Venue: arXiv cs.GT 2021
Year: 2021

Public goods games in undirected networks are generally known to have pure Nash equilibria, which are easy to find. In contrast, we prove that, in directed networks, a broad range of public goods games have intractable equilibrium problems: The existence of pure Nash equilibria is NP-hard to decide, and mixed Nash equilibria are PPAD-hard to find. We define general utility public goods games, and prove a complexity dichotomy result for finding pure equilibria, and a PPAD-completeness proof for mixed Nash equilibria. Even in the divisible goods variant of the problem, where existence is easy to prove, finding the equilibrium is PPAD-complete. Finally, when the treewidth of the directed network is appropriately bounded, we prove that polynomial-time algorithms are possible.
Stochastic Collapsed Variational Inference for Structured Gaussian Process Regression Network
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents an efficient variational inference framework for deriving a family of structured gaussian process regression network (SGPRN) models. The key idea is to incorporate auxiliary inducing variables in latent functions and jointly treats both the distributions of the inducing variables and hyper-parameters as variational parameters. Then we propose structured variable distributions and marginalize latent variables, which enables the decomposability of a tractable variational lower bound and leads to stochastic optimization. Our inference approach is able to model data in which outputs do not share a common input set with a computational complexity independent of the size of the inputs and outputs and thus easily handle datasets with missing values. We illustrate the performance of our method on synthetic data and real datasets and show that our model generally provides better imputation results on missing data than the state-of-the-art. We also provide a visualization approach for time-varying correlation across outputs in electrocoticography data and those estimates provide insight to understand the neural population dynamics.
Enabling Efficiency-Precision Trade-offs for Label Trees in Extreme Classification
Venue: arXiv cs.LG 2021
Year: 2021

Extreme multi-label classification (XMC) aims to learn a model that can tag data points with a subset of relevant labels from an extremely large label set. Real world e-commerce applications like personalized recommendations and product advertising can be formulated as XMC problems, where the objective is to predict for a user a small subset of items from a catalog of several million products. For such applications, a common approach is to organize these labels into a tree, enabling training and inference times that are logarithmic in the number of labels. While training a model once a label tree is available is well studied, designing the structure of the tree is a difficult task that is not yet well understood, and can dramatically impact both model latency and statistical performance. Existing approaches to tree construction fall at an extreme point, either optimizing exclusively for statistical performance, or for latency. We propose an efficient information theory inspired algorithm to construct intermediary operating points that trade off between the benefits of both. Our algorithm enables interpolation between these objectives, which was not previously possible. We corroborate our theoretical analysis with numerical results, showing that on the Wiki-500K benchmark dataset our method can reduce a proxy for expected latency by up to 28% while maintaining the same accuracy as Parabel. On several datasets derived from e-commerce customer logs, our modified label tree is able to improve this expected latency metric by up to 20% while maintaining the same accuracy. Finally, we discuss challenges in realizing these latency improvements in deployed models.
SWIPT with Intelligent Reflecting Surfaces under Spatial Correlation
Venue: arXiv cs.IT 2021
Year: 2021

Intelligent reflecting surfaces (IRSs) can be beneficial to both information and energy transfer, due to the gains achieved by their multiple elements. In this work, we deal with the impact of spatial correlation between the IRS elements, in the context of simultaneous wireless information and power transfer. The performance is evaluated in terms of the average harvested energy and the outage probability for random and equal phase shifts. Closed-form analytical expressions for both metrics under spatial correlation are derived. Moreover, the optimal case is considered when the elements are uncorrelated and fully correlated. In the uncorrelated case, random and equal phase shifts provide the same performance. However, the performance of correlated elements attains significant gains when there are equal phase shifts. Finally, we show that correlation is always beneficial to energy transfer, whereas it is a degrading factor for information transfer under random and optimal configurations.
Information Theoretic Measures for Fairness-aware Feature Selection
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning algorithms are increasingly used for consequential decision making regarding individuals based on their relevant features. Features that are relevant for accurate decisions may however lead to either explicit or implicit forms of discrimination against unprivileged groups, such as those of certain race or gender. This happens due to existing biases in the training data, which are often replicated or even exacerbated by the learning algorithm. Identifying and measuring these biases at the data level is a challenging problem due to the interdependence among the features, and the decision outcome. In this work, we develop a framework for fairness-aware feature selection which takes into account the correlation among the features and the decision outcome, and is based on information theoretic measures for the accuracy and discriminatory impacts of features. In particular, we first propose information theoretic measures which quantify the impact of different subsets of features on the accuracy and discrimination of the decision outcomes. We then deduce the marginal impact of each feature using Shapley value function; a solution concept in cooperative game theory used to estimate marginal contributions of players in a coalitional game. Finally, we design a fairness utility score for each feature (for feature selection) which quantifies how this feature influences accurate as well as nondiscriminatory decisions. Our framework depends on the joint statistics of the data rather than a particular classifier design. We examine our proposed framework on real and synthetic data to evaluate its performance.
Multi-task fully convolutional network for tree species mapping in dense forests using small training hyperspectral data
Venue: arXiv cs.CV 2021
Year: 2021

This work proposes a multi-task fully convolutional architecture for tree species mapping in dense forests from sparse and scarce polygon-level annotations using hyperspectral UAV-borne data. Our model implements a partial loss function that enables dense tree semantic labeling outcomes from non-dense training samples, and a distance regression complementary task that enforces tree crown boundary constraints and substantially improves the model performance. Our multi-task architecture uses a shared backbone network that learns common representations for both tasks and two task-specific decoders, one for the semantic segmentation output and one for the distance map regression. We report that introducing the complementary task boosts the semantic segmentation performance compared to the single-task counterpart in up to 11% reaching an average user's accuracy of 88.63% and an average producer's accuracy of 88.59%, achieving state-of-art performance for tree species classification in tropical forests.
On stimulating fungi $Pleurotus~ostreatus$ with Cortisol
Venue: arXiv cs.ET 2021
Year: 2021

Fungi cells are capable of sensing extracellular cues through reception, transduction and response systems which allow them to communicate with their host and adapt to their environment. They display effective regulatory protein expressions which enhance and regulate their response and adaptation to a variety of triggers such as stress, hormones, light, chemicals and host factors. In our recent studies, we have shown that $Pleurotus$ oyster fungi generate electrical potential impulses in the form of spike events as a result of their exposure to environmental, mechanical and chemical triggers, demonstrating that it is possible to discern the nature of stimuli from the fungi electrical responses. Harnessing the power of fungi sensing and intelligent capabilities, we explored the communication protocols of fungi as reporters of human chemical secretions such as hormones, addressing the question if fungi can sense human signals. We exposed $Pleurotus$ oyster fungi to cortisol, directly applied to a surface of a hemp shavings substrate colonised by fungi, and recorded the electrical activity of fungi. The response of fungi to cortisol was also supplementary studied through the application of X-ray to identify changes in the fungi tissue, where receiving cortisol by the substrate can inhibit the flow of calcium and, in turn, reduce its physiological changes. This study could pave the way for future research on adaptive fungal wearables capable for detecting physiological states of humans and biosensors made of living fungi.
An Optimal Relay Scheme for Outage Minimization in Fog-based Internet-of-Things (IoT) Networks
Venue: arXiv cs.NI 2021
Year: 2021

Fog devices are beginning to play a key role in relaying data and services within the Internet-of-Things (IoT) ecosystem. These relays may be static or mobile, with the latter offering a new degree of freedom for performance improvement via careful relay mobility design. Besides that, power conservation has been a prevalent issue in IoT networks with devices being power-constrained, requiring optimal power-control mechanisms. In this paper, we consider a multi-tier fog-based IoT architecture where a mobile/static fog node acts as an amplify and forward relay that transmits received information from a sensor node to a higher hierarchically-placed static fog device, which offers some localized services. The outage probability of the presented scenario was efficiently minimized by jointly optimizing the mobility pattern and the transmit power of the fog relay. A closed-form analytical expression for the outage probability was derived. Furthermore, due to the intractability and non-convexity of the formulated problem, we applied an iterative algorithm based on the steepest descent method to arrive at a desirable objective. Simulations reveal that the outage probability was improved by 62.7% in the optimized-location fixed-power (OLFP) scheme, 79.3% in the optimized-power fixed-location (OPFL) scheme, and 94.2% in the optimized-location optimized-power (OLOP) scheme, as against the fixed-location and fixed-power (FLFP) scheme (i.e., without optimization). Lastly, we present an optimal relay selection strategy that chooses an appropriate relay node from randomly distributed relaying candidates.