OpenRaiser/Intern-Atlas huggingface dataset:
SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognition
Venue: arXiv cs.CV 2023
Year: 2023

Fully supervised skeleton-based action recognition has achieved great progress with the blooming of deep learning techniques. However, these methods require sufficient labeled data which is not easy to obtain. In contrast, self-supervised skeleton-based action recognition has attracted more attention. With utilizing the unlabeled data, more generalizable features can be learned to alleviate the overfitting problem and reduce the demand of massive labeled training data. Inspired by the MAE, we propose a spatial-temporal masked autoencoder framework for self-supervised 3D skeleton-based action recognition (SkeletonMAE). Following MAE's masking and reconstruction pipeline, we utilize a skeleton-based encoder-decoder transformer architecture to reconstruct the masked skeleton sequences. A novel masking strategy, named Spatial-Temporal Masking, is introduced in terms of both joint-level and frame-level for the skeleton sequence. This pre-training strategy makes the encoder output generalizable skeleton features with spatial and temporal dependencies. Given the unmasked skeleton sequence, the encoder is fine-tuned for the action recognition task. Extensive experiments show that our SkeletonMAE achieves remarkable performance and outperforms the state-of-the-art methods on both NTU RGB+D and NTU RGB+D 120 datasets.
CausalRCA: Causal Inference based Precise Fine-grained Root Cause Localization for Microservice Applications
Venue: arXiv cs.DC 2023
Year: 2023

Effectively localizing root causes of performance anomalies is crucial to enabling the rapid recovery and loss mitigation of microservice applications in the cloud. Depending on the granularity of the causes that can be localized, a service operator may take different actions, e.g., restarting or migrating services if only faulty services can be localized (namely, coarse-grained) or scaling resources if specific indicative metrics on the faulty service can be localized (namely, fine-grained). Prior research mainly focuses on coarse-grained faulty service localization, and there is now a growing interest in fine-grained root cause localization to identify faulty services and metrics. Causal inference (CI) based methods have gained popularity recently for root cause localization, but currently used CI methods have limitations, such as the linear causal relations assumption and strict data distribution requirements. To tackle these challenges, we propose a framework named CausalRCA to implement fine-grained, automated, and real-time root cause localization. The CausalRCA uses a gradient-based causal structure learning method to generate weighted causal graphs and a root cause inference method to localize root cause metrics. We conduct coarse- and fine-grained root cause localization to evaluate the localization performance of CausalRCA. Experimental results show that CausalRCA has significantly outperformed baseline methods in localization accuracy, e.g., the average AC@3 of the fine-grained root cause metric localization in the faulty service is 0.719, and the average increase is 10% compared with baseline methods. In addition, the average Avg@5 has improved by 9.43%.
Unifying Generative Models with GFlowNets and Beyond
Venue: arXiv cs.LG 2023
Year: 2023

There are many frameworks for deep generative modeling, each often presented with their own specific training algorithms and inference methods. Here, we demonstrate the connections between existing deep generative models and the recently introduced GFlowNet framework, a probabilistic inference machine which treats sampling as a decision-making process. This analysis sheds light on their overlapping traits and provides a unifying viewpoint through the lens of learning with Markovian trajectories. Our framework provides a means for unifying training and inference algorithms, and provides a route to shine a unifying light over many generative models. Beyond this, we provide a practical and experimentally verified recipe for improving generative modeling with insights from the GFlowNet perspective.
Play&Go Corporate: An End-to-End Solution for Facilitating Urban Cyclability
Venue: arXiv cs.CY 2023
Year: 2023

Mobility plays a fundamental role in modern cities. How citizens experience the urban environment, access city core services, and participate in city life, strongly depends on its mobility organization and efficiency. The challenges that municipalities face are very ambitious: on the one hand, administrators must guarantee their citizens the right to mobility and to easily access local services; on the other hand, they need to minimize the economic, social, and environmental costs of the mobility system. Municipalities are increasingly facing problems of traffic congestion, road safety, energy dependency and air pollution, and therefore encouraging a shift towards sustainable mobility habits based on active mobility is of central importance. Active modes, such as cycling, should be particularly encouraged, especially for local recurrent journeys (e.g., home--to--school, home--to--work). In this context, addressing and mitigating commuter-generated traffic requires engaging public and private stakeholders through innovative and collaborative approaches that focus not only on supply (e.g., roads and vehicles) but also on transportation demand management. In this paper, we present an end-to-end solution, called Play&Go Corporate, for enabling urban cyclability and its concrete exploitation in the realization of a home-to-work sustainable mobility campaign (i.e., Bike2Work) targeting employees of public and private companies. To evaluate the effectiveness of the proposed solution we developed two analyses: the first to carefully analyze the user experience and any behaviour change related to the Bike2Work mobility campaign, and the second to demonstrate how exploiting the collected data we can potentially inform and guide the involved municipality (i.e., Ferrara, a city in Northern Italy) in improving urban cyclability.
Security and Privacy of IP-ICN Coexistence: A Comprehensive Survey
Venue: arXiv cs.CR 2023
Year: 2023

Today Internet is experiencing a massive number of users with a continuously increasing need for data, which is the leading cause of introduced limitations among security and privacy issues. To overcome these limitations, a shift from host-centric to data-centric is proposed, and in this context, Information-Centric Networking (ICN) represents a promising solution. Nevertheless, unsettling the current Internet network layer, i.e., Internet Protocol (IP), with ICN is a challenging, expensive task since it requires worldwide coordination among Internet Service Providers (ISPs), backbone, and Autonomous Services (AS). Therefore, researchers foresee that the replacement process of the current Internet will transition through the coexistence of IP and ICN. In this perspective, novel architectures combine IP and ICN protocols. However, only a few of the proposed architectures place the security-by-design feature. Therefore, this article provides the first comprehensive Security and Privacy (SP) analysis of the state-of-the-art IP-ICN coexistence architectures by horizontally comparing the SP features among three deployment approaches, i.e., overlay, underlay, and hybrid, and vertically comparing among the ten considered SP features. Lastly, the article sheds light on the open issues and possible future directions for IP-ICN coexistence. Our analysis shows that most architectures fail to provide several SP features, including data and traffic flow confidentiality, availability, and anonymity of communication. Thus, this article shows the secure combination of current and future protocol stacks during the coexistence phase that the Internet will definitely walk across.
Multimodal learning with graphs
Venue: arXiv cs.LG 2023
Year: 2023

Artificial intelligence for graphs has achieved remarkable success in modeling complex systems, ranging from dynamic networks in biology to interacting particle systems in physics. However, the increasingly heterogeneous graph datasets call for multimodal methods that can combine different inductive biases: the set of assumptions that algorithms use to make predictions for inputs they have not encountered during training. Learning on multimodal datasets presents fundamental challenges because the inductive biases can vary by data modality and graphs might not be explicitly given in the input. To address these challenges, multimodal graph AI methods combine different modalities while leveraging cross-modal dependencies using graphs. Diverse datasets are combined using graphs and fed into sophisticated multimodal architectures, specified as image-intensive, knowledge-grounded and language-intensive models. Using this categorization, we introduce a blueprint for multimodal graph learning, use it to study existing methods and provide guidelines to design new models.
Quantifying Aleatoric and Epistemic Uncertainty in Machine Learning: Are Conditional Entropy and Mutual Information Appropriate Measures?
Venue: arXiv cs.LG 2023
Year: 2023

The quantification of aleatoric and epistemic uncertainty in terms of conditional entropy and mutual information, respectively, has recently become quite common in machine learning. While the properties of these measures, which are rooted in information theory, seem appealing at first glance, we identify various incoherencies that call their appropriateness into question. In addition to the measures themselves, we critically discuss the idea of an additive decomposition of total uncertainty into its aleatoric and epistemic constituents. Experiments across different computer vision tasks support our theoretical findings and raise concerns about current practice in uncertainty quantification.
A simple approach for quantizing neural networks
Venue: arXiv cs.LG 2023
Year: 2023

In this short note, we propose a new method for quantizing the weights of a fully trained neural network. A simple deterministic pre-processing step allows us to quantize network layers via memoryless scalar quantization while preserving the network performance on given training data. On one hand, the computational complexity of this pre-processing slightly exceeds that of state-of-the-art algorithms in the literature. On the other hand, our approach does not require any hyper-parameter tuning and, in contrast to previous methods, allows a plain analysis. We provide rigorous theoretical guarantees in the case of quantizing single network layers and show that the relative error decays with the number of parameters in the network if the training data behaves well, e.g., if it is sampled from suitable random distributions. The developed method also readily allows the quantization of deep networks by consecutive application to single layers.
A Secure and Efficient Multi-Object Grasping Detection Approach for Robotic Arms
Venue: arXiv cs.RO 2023
Year: 2023

Robotic arms are widely used in automatic industries. However, with wide applications of deep learning in robotic arms, there are new challenges such as the allocation of grasping computing power and the growing demand for security. In this work, we propose a robotic arm grasping approach based on deep learning and edge-cloud collaboration. This approach realizes the arbitrary grasp planning of the robot arm and considers the grasp efficiency and information security. In addition, the encoder and decoder trained by GAN enable the images to be encrypted while compressing, which ensures the security of privacy. The model achieves 92% accuracy on the OCID dataset, the image compression ratio reaches 0.03%, and the structural difference value is higher than 0.91.
GoonDAE: Denoising-Based Driver Assistance for Off-Road Teleoperation
Venue: arXiv cs.RO 2023
Year: 2023

Because of the limitations of autonomous driving technologies, teleoperation is widely used in dangerous environments such as military operations. However, the teleoperated driving performance depends considerably on the driver's skill level. Moreover, unskilled drivers need extensive training time for teleoperations in unusual and harsh environments. To address this problem, we propose a novel denoising-based driver assistance method, namely GoonDAE, for real-time teleoperated off-road driving. The unskilled driver control input is assumed to be the same as the skilled driver control input but with noise. We designed a skip-connected long short-term memory (LSTM)-based denoising autoencoder (DAE) model to assist the unskilled driver control input by denoising. The proposed GoonDAE was trained with skilled driver control input and sensor data collected from our simulated off-road driving environment. To evaluate GoonDAE, we conducted an experiment with unskilled drivers in the simulated environment. The results revealed that the proposed system considerably enhanced driving performance in terms of driving stability.
Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes
Venue: arXiv cs.LG 2023
Year: 2023

A fundamental property of deep learning normalization techniques, such as batch normalization, is making the pre-normalization parameters scale invariant. The intrinsic domain of such parameters is the unit sphere, and therefore their gradient optimization dynamics can be represented via spherical optimization with varying effective learning rate (ELR), which was studied previously. However, the varying ELR may obscure certain characteristics of the intrinsic loss landscape structure. In this work, we investigate the properties of training scale-invariant neural networks directly on the sphere using a fixed ELR. We discover three regimes of such training depending on the ELR value: convergence, chaotic equilibrium, and divergence. We study these regimes in detail both on a theoretical examination of a toy example and on a thorough empirical analysis of real scale-invariant deep learning models. Each regime has unique features and reflects specific properties of the intrinsic loss landscape, some of which have strong parallels with previous research on both regular and scale-invariant neural networks training. Finally, we demonstrate how the discovered regimes are reflected in conventional training of normalized networks and how they can be leveraged to achieve better optima.
Fact-Saboteurs: A Taxonomy of Evidence Manipulation Attacks against Fact-Verification Systems
Venue: arXiv cs.CR 2023
Year: 2023

Mis- and disinformation are a substantial global threat to our security and safety. To cope with the scale of online misinformation, researchers have been working on automating fact-checking by retrieving and verifying against relevant evidence. However, despite many advances, a comprehensive evaluation of the possible attack vectors against such systems is still lacking. Particularly, the automated fact-verification process might be vulnerable to the exact disinformation campaigns it is trying to combat. In this work, we assume an adversary that automatically tampers with the online evidence in order to disrupt the fact-checking model via camouflaging the relevant evidence or planting a misleading one. We first propose an exploratory taxonomy that spans these two targets and the different threat model dimensions. Guided by this, we design and propose several potential attack methods. We show that it is possible to subtly modify claim-salient snippets in the evidence and generate diverse and claim-aligned evidence. Thus, we highly degrade the fact-checking performance under many different permutations of the taxonomy's dimensions. The attacks are also robust against post-hoc modifications of the claim. Our analysis further hints at potential limitations in models' inference when faced with contradicting evidence. We emphasize that these attacks can have harmful implications on the inspectable and human-in-the-loop usage scenarios of such models, and we conclude by discussing challenges and directions for future defenses.
Hardware-Software Co-Design of BIKE with HLS-Generated Accelerators
Venue: arXiv cs.AR 2023
Year: 2023

In order to mitigate the security threat of quantum computers, NIST is undertaking a process to standardize post-quantum cryptosystems, aiming to assess their security and speed up their adoption in production scenarios. Several hardware and software implementations have been proposed for each candidate, while only a few target heterogeneous platforms featuring CPUs and FPGAs. This work presents a HW/SW co-design of BIKE for embedded platforms featuring both CPUs and small FPGAs and employs high-level synthesis (HLS) to timely deliver the hardware accelerators. In contrast to state-of-the-art solutions targeting performance-optimized HLS accelerators, the proposed solution targets the small FPGAs implemented in the heterogeneous platforms for embedded systems. Compared to the software-only execution of BIKE, the experimental results collected on the systems-on-chip of the entire Xilinx Zynq-7000 family highlight a performance speedup ranging from 1.37x, on Z-7010, to 2.78x, on Z-7020.
FADE: Enabling Federated Adversarial Training on Heterogeneous Resource-Constrained Edge Devices
Venue: arXiv cs.LG 2023
Year: 2023

Federated adversarial training can effectively complement adversarial robustness into the privacy-preserving federated learning systems. However, the high demand for memory capacity and computing power makes large-scale federated adversarial training infeasible on resource-constrained edge devices. Few previous studies in federated adversarial training have tried to tackle both memory and computational constraints simultaneously. In this paper, we propose a new framework named Federated Adversarial Decoupled Learning (FADE) to enable AT on heterogeneous resource-constrained edge devices. FADE differentially decouples the entire model into small modules to fit into the resource budget of each device, and each device only needs to perform AT on a single module in each communication round. We also propose an auxiliary weight decay to alleviate objective inconsistency and achieve better accuracy-robustness balance in FADE. FADE offers theoretical guarantees for convergence and adversarial robustness, and our experimental results show that FADE can significantly reduce the consumption of memory and computing power while maintaining accuracy and robustness.
Learning Sparse Graphon Mean Field Games
Venue: arXiv cs.MA 2023
Year: 2023

Although the field of multi-agent reinforcement learning (MARL) has made considerable progress in the last years, solving systems with a large number of agents remains a hard challenge. Graphon mean field games (GMFGs) enable the scalable analysis of MARL problems that are otherwise intractable. By the mathematical structure of graphons, this approach is limited to dense graphs which are insufficient to describe many real-world networks such as power law graphs. Our paper introduces a novel formulation of GMFGs, called LPGMFGs, which leverages the graph theoretical concept of $L^p$ graphons and provides a machine learning tool to efficiently and accurately approximate solutions for sparse network problems. This especially includes power law networks which are empirically observed in various application areas and cannot be captured by standard graphons. We derive theoretical existence and convergence guarantees and give empirical examples that demonstrate the accuracy of our learning approach for systems with many agents. Furthermore, we extend the Online Mirror Descent (OMD) learning algorithm to our setup to accelerate learning speed, empirically show its capabilities, and conduct a theoretical analysis using the novel concept of smoothed step graphons. In general, we provide a scalable, mathematically well-founded machine learning approach to a large class of otherwise intractable problems of great relevance in numerous research fields.
Trajectory Range Visibility
Venue: arXiv cs.CG 2023
Year: 2023

Consider two entities with constant but not necessarily equal velocities, moving on two given piece-wise linear trajectories inside a simple polygon $P$. The Trajectory Range Visibility problem deals with determining the sub-trajectories on which two entities become visible to each other. A more straightforward decision version of this problem is called Trajectory Visibility, where the trajectories are line segments. The decision version specifies whether the entities can see one another. This version was studied by P. Eades et al. in 2020, where they supposed given constant velocities for the entities. However, the approach presented in this paper supports non-constant complexity trajectories. Furthermore, we report every pair of constant velocities with which the entities can see each other. In particular, for every constant velocity of a moving entity, we specify: $(1)$ All visible parts of the other entity's trajectory. $(2)$ All possible constant velocities of the other entity to become visible. Regarding line-segment trajectories, we present $\mathcal{O}(n \log n)$ running time algorithm which obtains all pairs of sub-trajectories on which the moving entities become visible to one another, where $n$ is the complexity of $P$. Regarding the general case, we provide an algorithm with $\mathcal{O}(n \log n + m(\log m + \log n))$ running time, where $m$ indicates the complexity of both trajectories. We offer $\mathcal{O}(\log n)$ query time for line segment trajectories and $\mathcal{O}(\log m + k)$ for the non-constant complexity ones s.t. $k$ is the number of velocity ranges reported in the output. Interestingly, our results require only $\mathcal{O}(n + m)$ space for non-constant complexity trajectories.
Causal Modeling of Policy Interventions From Sequences of Treatments and Outcomes
Venue: arXiv cs.LG 2023
Year: 2023

A treatment policy defines when and what treatments are applied to affect some outcome of interest. Data-driven decision-making requires the ability to predict what happens if a policy is changed. Existing methods that predict how the outcome evolves under different scenarios assume that the tentative sequences of future treatments are fixed in advance, while in practice the treatments are determined stochastically by a policy and may depend, for example, on the efficiency of previous treatments. Therefore, the current methods are not applicable if the treatment policy is unknown or a counterfactual analysis is needed. To handle these limitations, we model the treatments and outcomes jointly in continuous time, by combining Gaussian processes and point processes. Our model enables the estimation of a treatment policy from observational sequences of treatments and outcomes, and it can predict the interventional and counterfactual progression of the outcome after an intervention on the treatment policy (in contrast with the causal effect of a single treatment). We show with real-world and semi-synthetic data on blood glucose progression that our method can answer causal queries more accurately than existing alternatives.
A Close Look at a Systematic Method for Analyzing Sets of Security Advice
Venue: arXiv cs.CR 2023
Year: 2023

We carry out a detailed analysis of the security advice coding method (SAcoding) of Barrera et al. (2021), which is designed to analyze security advice in the sense of measuring actionability and categorizing advice items as practices, policies, principles, or outcomes. The main part of our analysis explores the extent to which a second coder's assignment of codes to advice items agrees with that of a first, for a dataset of 1013 security advice items nominally addressing Internet of Things devices. More broadly, we seek a deeper understanding of the soundness and utility of the SAcoding method, and the degree to which it meets the design goal of reducing subjectivity in assigning codes to security advice items. Our analysis results in suggestions for modifications to the coding tree methodology, and some recommendations. We believe the coding tree approach may be of interest for analysis of qualitative data beyond security advice datasets alone.