Miracle-dz/UniFAR-FTUs huggingface dataset:
Spectral Temporal Graph Neural Network for Multivariate Time-series Forecasting

Multivariate time-series forecasting plays a crucial role in many real-world applications. It is a challenging problem as one needs to consider both intra-series temporal correlations and inter-series correlations simultaneously. Recently, there have been multiple works trying to capture both correlations, but most, if not all of them only capture temporal correlations in the time domain and resort to pre-defined priors as inter-series relationships. In this paper, we propose Spectral Temporal Graph Neural Network (StemGNN) to further improve the accuracy of multivariate time-series forecasting. StemGNN captures inter-series correlations and temporal dependencies jointly in the spectral domain. It combines Graph Fourier Transform (GFT) which models inter-series correlations and Discrete Fourier Transform (DFT) which models temporal dependencies in an end-to-end framework. After passing through GFT and DFT, the spectral representations hold clear patterns and can be predicted effectively by convolution and sequential learning modules. Moreover, StemGNN learns inter-series correlations automatically from the data without using pre-defined priors. We conduct extensive experiments on ten real-world datasets to demonstrate the effectiveness of StemGNN.
Every View Counts: Cross-View Consistency in 3D Object Detection with Hybrid-Cylindrical-Spherical Voxelization

Recent voxel-based 3D object detectors for autonomous vehicles learn point cloud representations either from bird eye view (BEV) or range view (RV, a.k.a. perspective view). However, each view has its own strengths and weaknesses. In this paper, we present a novel framework to unify and leverage the beneﬁts from both BEV and RV. The widely-used cuboid-shaped voxels in Cartesian coordinate system only beneﬁt BEV feature map. Therefore, to enable learning both BEV and RV feature maps, we introduce Hybrid-Cylindrical-Spherical voxelization. Our ﬁndings show that simply adding detection on another view as auxiliary supervision will lead to poor performance. We proposed a pair of cross-view transformers to transform the feature maps into the other view and introduce cross-view consistency loss on them. Comprehensive experiments on the challenging NuScenes Dataset validate the effectiveness of our proposed method which leverages joint optimization and complementary information on both views. Remarkably, our approach achieved mAP of 55 . 8% , outperforming all published approaches by at least 3% in overall performance and up to 16 . 5% in safety-crucial categories like cyclist.
Online Adaptation for Consistent Mesh Reconstruction in the Wild

This paper presents an algorithm to reconstruct temporally consistent 3D meshes of deformable object instances from videos in the wild. Without requiring annotations of 3D mesh, 2D keypoints, or camera pose for each video frame, we pose video-based reconstruction as a self-supervised online adaptation problem applied to any incoming test video. We first learn a category-specific 3D reconstruction model from a collection of single-view images of the same category that jointly predicts the shape, texture, and camera pose of an image. Then, at inference time, we adapt the model to a test video over time using self-supervised regularization terms that exploit temporal consistency of an object instance to enforce that all reconstructed meshes share a common texture map, a base shape, as well as parts. We demonstrate that our algorithm recovers temporally consistent and reliable 3D structures from videos of non-rigid objects including those of animals captured in the wild -- an extremely challenging task rarely addressed before.
Matrix inference and estimation in multi-layer models

We consider the problem of estimating the input and hidden variables of a stochastic multi-layer neural network (NN) from an observation of the output. The hidden variables in each layer are represented as matrices with statistical interactions along both rows as well as columns. This problem applies to matrix imputation, signal recovery via deep generative prior models, multi-task and mixed regression, and learning certain classes of two-layer NNs. We extend a recently-developed algorithm—multi-layer vector approximate message passing, for this matrix-valued inference problem. It is shown that the performance of the proposed multi-layer matrix vector approximate message passing algorithm can be exactly predicted in a certain random large-system limit, where the dimensions N × d of the unknown quantities grow as N → ∞ with d fixed. In the two-layer neural-network learning problem, this scaling corresponds to the case where the number of input features as well as training samples grow to infinity but the number of hidden nodes stays fixed. The analysis enables a precise prediction of the parameter and test error of the learning.
An Empirical Analysis of Test Input Generation Tools for Android Apps through a Sequence of Events

Graphical User Interface (GUI) testing of Android apps has gained considerable interest from the industries and research community due to its excellent capability to verify the operational requirements of GUI components. To date, most of the existing GUI testing tools for Android apps are capable of generating test inputs by using different approaches and improve the Android apps’ code coverage and fault detection performance. Many previous studies have evaluated the code coverage and crash detection performances of GUI testing tools in the literature. However, very few studies have investigated the effectiveness of the test input generation tools, especially in the events sequence length of the overall test coverage and crash detection. The event sequence length generally shows the number of steps required by the test input generation tools to detect a crash. It is critical to highlight its effectiveness due to its significant effects on time, testing effort, and computational cost. Thus, this study evaluated the effectiveness of six test input generation tools for Android apps that support the system events generation on 50 Android apps. The generation tools were evaluated and compared based on the activity coverage, method coverage, and capability in detecting crashes. Through a critical analysis of the results, this study identifies the diversity and similarity of test input generation tools for Android apps to provide a clear picture of the current state of the art. The results revealed that a long events sequence performed better than a shorter events sequence. However, a long events sequence led to a minor positive effect on the coverage and crash detection. Moreover, the study showed that the tools achieved less than 40% of the method coverage and 67% of the activity coverage.
Approximate Heavily-Constrained Learning with Lagrange Multiplier Models

In machine learning applications such as ranking fairness or fairness over intersec-tional groups, one often encounters optimization problems with extremely large numbers of constraints. In particular, with ranking fairness tasks, there may even be a variable number of constraints, e.g. one for each query in the training set. In these cases, the standard approach of optimizing a Lagrangian while maintaining one Lagrange multiplier per constraint may no longer be practical. Our proposal is to associate a feature vector with each constraint, and to learn a “multiplier model” that maps each such vector to the corresponding Lagrange multiplier. We prove optimality, approximate feasibility and generalization guarantees under assumptions on the ﬂexibility of the multiplier model, and empirically demonstrate that our method is effective on real-world case studies.
Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Evolvability

In this paper, we revisit the problem of distribution-independently learning half-spaces under Massart noise with rate ⌘ . Recent work [DGT19] resolved a long-standing problem in this model of efﬁciently learning to error ⌘ + ✏ for any ✏ > 0 , by giving an improper learner that partitions space into poly( d, 1 / ✏ ) regions. Here we give a much simpler algorithm and settle a number of outstanding open questions: (1) We give the ﬁrst proper learner for Massart halfspaces that achieves ⌘ + ✏ . (2) Based on (1), we develop a blackbox knowledge distillation procedure to convert an arbitrarily complex classiﬁer to an equally good proper classiﬁer. (3) By leveraging a simple but overlooked connection to evolvability , we show any SQ algorithm requires super-polynomially many queries to achieve OPT + ✏ . We then zoom out to study generalized linear models and give an efﬁcient algorithm for learning under a challenging new corruption model generalizing Massart noise. Lastly, we empirically evaluate our algorithm for Massart halfspaces and ﬁnd it exhibits some intriguing fairness properties.
Resting EEG Asymmetry Markers of Multiple Facets of the Behavioral Approach System: A LORETA Analysis

Previously published models of frontal activity linked high relative left frontal activity to the behavioral approach system (BAS) and impulsivity. Additionally, these models did not account for BAS facets encompassing the anticipation of reward, i.e., goal-driven persistence (BAS–GDP) and reward interest (BAS–RI), from those that deal with the actual hedonic experience of reward, i.e., reward reactivity (BAS–RR) and impulsivity (BAS–I). Using resting electroencephalographic (EEG) recordings, the source localization (LORETA) method allowed us to calculate the hemispheric asymmetry of the current density within the alpha band (7.5–13 Hz) in ten regions of interest. Compared to low BAS subtrait scorers, high BAS subtrait scorers (except for BAS–I) were correlated with greater relative left-sided activity in the superior frontal gyrus (BA10). Further, an isolated effective coherence (iCOH) analysis of the beta activity (21 Hz) disclosed that high impulsive scorers as compared to low impulsive ones had higher connectivity between the superior frontal gyrus and middle temporal gyrus, which was not compensated for by enhanced inhibitory alpha (11 Hz) connectivity between these regions. For the beta frequency, we also found in highly impulsive individuals that (i) both left and right middle temporal lobes directly influenced the activity of the left and right superior frontal lobes, and (ii) a clear decoupling between left and right superior frontal lobes. These findings could indicate reduced control by the supervisory system in more impulsive individuals.
Agnostic Learning with Multiple Objectives

Most machine learning tasks are inherently multi-objective. This means that the learner has to come up with a model that performs well across a number of base objectives L 1 , . , L p , as opposed to a single one. Since optimizing with respect to multiple objectives at the same time is often computationally expensive, the base objectives are often combined in an ensemble (cid:80) pk =1 λ k L k , thereby reducing the problem to scalar optimization. The mixture weights λ k are set to uniform or some other ﬁxed distribution, based on the learner’s preferences. We argue that learning with a ﬁxed distribution on the mixture weights runs the risk of overﬁtting to some individual objectives and signiﬁcantly harming others, despite performing well on an entire ensemble. Moreover, in reality, the true preferences of a learner across multiple objectives are often unknown or hard to express as a speciﬁc distribution. Instead, we propose a new framework of Agnostic Learning with Multiple Objectives (ALMO), where a model is optimized for any weights in the mixture of base objectives. We present data-dependent Rademacher complexity guarantees for learning in the ALMO framework, which are used to guide a scalable optimization algorithm and the corresponding regularization. We present convergence guarantees for this algorithm, assuming convexity of the loss functions and the underlying hypothesis space. We further implement the algorithm in a popular symbolic gradient computation framework and empirically demonstrate on a number of datasets the beneﬁts of ALMO framework versus learning with a ﬁxed mixture weights distribution.
Consistent Structural Relation Learning for Zero-Shot Segmentation

Zero-shot semantic segmentation aims to recognize the semantics of pixels from unseen categories with zero training samples. Previous practice [1] proposed to train the classiﬁers for unseen categories using the visual features generated from semantic word embeddings. However, the generator is merely learned on the seen categories while no constraint is applied to the unseen categories, leading to poor generalization ability. In this work, we propose a Consistent Structural Relation Learning (CSRL) approach to constrain the generating of unseen visual features by exploiting the structural relations between seen and unseen categories. We observe that different categories are usually with similar relations in either semantic word embedding space or visual feature space. This observation motivates us to harness the similarity of category-level relations on the semantic word embedding space to learn a better visual feature generator. Concretely, by exploring the pair-wise and list-wise structures, we impose the relations of generated visual features to be consistent with their counterparts in the semantic word embedding space. In this way, the relations between seen and unseen categories will be transferred to implicitly constrain the generator to produce relation-consistent unseen visual features. We conduct extensive experiments on Pascal-VOC and Pascal-Context benchmarks. The proposed CSRL outperforms existing state-of-the-art methods by a large margin, resulting in ~7-12% on Pascal-VOC and ~2-5% on Pascal-Context.
The Predictive Corridor: A Virtual Augmented Driving Assistance System for Teleoperated Autonomous Vehicles

Autonomous vehicles offer a driverless future, however, despite the rapid progress in ubiquitous technologies, human situational assessment continues to be required. For example, upon recognizing an obstacle on the road a request might be routed to a tele-operator, who can assess and manage the situation with the help of a dedicated workspace. A common solution to this problem is direct remote steering. Thereby a key problem in teleoperation is the time latency and low remote situational awareness. To solve this issue we present the Predictive Corridor (PC), a virtual augmented driving assistance system for teleoperated autonomous vehicles. In a user study (N =32), we evaluated the PC by employing three measures: performance, subjective and physiological measures. The results demonstrate that driving with the PC is less cognitively demanding, improves operational performance, and nonetheless can visually compensate for the effect of the time delay between the teleoperator and the vehicle. This technology, therefore, is promising for being applied in future teleoperation applications.
Variational Amodal Object Completion

In images of complex scenes, objects are often occluding each other which makes perception tasks such as object detection and tracking, or robotic control tasks such as planning, challenging. To facilitate downstream tasks, it is thus important to reason about the full extent of objects, i.e., seeing behind occlusion, typically referred to as amodal instance completion . In this paper, we propose a variational generative framework for amodal completion, referred to as Amodal-VAE, which does not require any amodal labels at training time, as it is able to utilize widely available object instance masks. We showcase our approach on the downstream task of scene editing where the user is presented with interactive tools to complete and erase objects in photographs. Experiments on complex street scenes demonstrate state-of-the-art performance in amodal mask completion, and showcase high quality scene editing results. Interestingly, a user study shows that humans prefer object completions inferred by our model to the human-labeled ones.
Optimal Learning from Verified Training Data

Standard machine learning algorithms typically assume that data is sampled independently from the distribution of interest. In attempts to relax this assumption, ﬁelds such as adversarial learning typically assume that data is provided by an adversary, whose sole objective is to fool a learning algorithm. However, in reality, it is often the case that data comes from self-interested agents, with less malicious goals and intentions which lie somewhere between the two settings described above. To tackle this problem, we present a Stackelberg competition model for least squares regression, in which data is provided by agents who wish to achieve speciﬁc predictions for their data. Although the resulting optimisation problem is nonconvex, we derive an algorithm which converges globally, outperforming current approaches which only guarantee convergence to local optima. We also provide empirical results on two real-world datasets, the medical personal costs dataset and the red wine dataset, showcasing the performance of our algorithm relative to algorithms which are optimal under adversarial assumptions, outperforming the state of the art.
Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach

In Federated Learning, we aim to train models across multiple computing units (users), while users can only communicate with a common central server, without exchanging their data samples. This mechanism exploits the computational power of all users and allows users to obtain a richer model as their models are trained over a larger set of data points. However, this scheme only develops a common output for all the users, and, therefore, it does not adapt the model to each user. This is an important missing feature, especially given the heterogeneity of the underlying data distribution for various users. In this paper, we study a personalized variant of the federated learning in which our goal is to ﬁnd an initial shared model that current or new users can easily adapt to their local dataset by performing one or a few steps of gradient descent with respect to their own data. This approach keeps all the beneﬁts of the federated learning architecture, and, by structure, leads to a more personalized model for each user. We show this problem can be studied within the Model-Agnostic Meta-Learning (MAML) framework. Inspired by this connection, we study a personalized variant of the well-known Federated Averaging algorithm and evaluate its performance in terms of gradient norm for non-convex loss functions. Further, we characterize how this performance is affected by the closeness of underlying distributions of user data, measured in terms of distribution distances such as Total Variation and 1-Wasserstein metric.
Auto Learning Attention

Attention modules have been demonstrated effective in strengthening the representation ability of a neural network via reweighting spatial or channel features or stacking both operations sequentially. However, designing the structures of different attention operations requires a bulk of computation and extensive expertise. In this paper, we devise an Auto Learning Attention (AutoLA) method, which is the ﬁrst attempt on automatic attention design. Speciﬁcally, we deﬁne a novel attention module named high order group attention (HOGA) as a directed acyclic graph (DAG) where each group represents a node, and each edge represents an operation of heterogeneous attentions. A typical HOGA architecture can be searched automatically via the differential AutoLA method within 1 GPU day using the ResNet-20 backbone on CIFAR10. Further, the searched attention module can generalize to various backbones as a plug-and-play component and outperforms popular manually designed channel and spatial attentions for many vision tasks, including image classiﬁcation on CIFAR100 and ImageNet, object detection and human keypoint detection on COCO dataset. Code is available at https://github.com/btma48/AutoLA.
On the Stability and Convergence of Robust Adversarial Reinforcement Learning: A Case Study on Linear Quadratic Systems

Reinforcement learning (RL) algorithms can fail to generalize due to the gap between the simulation and the real world. One standard remedy is to use robust adversarial RL (RARL) that accounts for this gap during the policy training, by modeling the gap as an adversary against the training agent. In this work, we reexamine the effectiveness of RARL under a fundamental robust control setting: the linear quadratic (LQ) case. We ﬁrst observe that the popular RARL scheme that greedily alternates agents’ updates can easily destabilize the system . Motivated by this, we propose several other policy-based RARL algorithms whose convergence behaviors are then studied both empirically and theoretically. We ﬁnd: i) the conventional RARL framework (Pinto et al., 2017) can learn a destabilizing policy if the initial policy does not enjoy the robust stability property against the adversary; and ii) with robustly stabilizing initializations, our proposed double-loop RARL algorithm provably converges to the global optimal cost while maintaining robust stability on-the-ﬂy. We also examine the stability and convergence issues of other variants of policy-based RARL algorithms, and then discuss several ways to learn robustly stabilizing initializations. From a robust control perspective, we aim to provide some new and critical angles about RARL, by identifying and addressing the stability issues in this fundamental LQ setting in continuous control. Our results make an initial attempt toward better theoretical understandings of policy-based RARL, the core approach in Pinto et al., 2017.
Assisted Learning: A Framework for Multi-Organization Learning

In an increasing number of AI scenarios, collaborations among different organizations or agents (e.g., human and robots, mobile units) are often essential to accomplish an organization-speciﬁc mission. However, to avoid leaking useful and possibly proprietary information, organizations typically enforce stringent security constraints on sharing modeling algorithms and data, which signiﬁcantly limits collaborations. In this work, we introduce the Assisted Learning framework for organizations to assist each other in supervised learning tasks without revealing any organization’s algorithm, data, or even task. An organization seeks assistance by broadcasting task-speciﬁc but nonsensitive statistics and incorporating others’ feedback in one or more iterations to eventually improve its predictive performance. Theoretical and experimental studies, including real-world medical benchmarks, show that Assisted Learning can often achieve near-oracle learning performance as if data and training processes were centralized.
A New Sum of Squares Exponentially Weighted Moving Average Control Chart Using Auxiliary Information

The concept of control charts is based on mathematics and statistics to process forecast; which applications are widely used in industrial management. The sum of squares exponentially weighted moving average (SSEWMA) chart is a well-known tool for effectively monitoring both the increase and decrease in the process mean and/or variability. In this paper, we propose a novel SSEWMA chart using auxiliary information, called the AIB-SSEWMA chart, for jointly monitoring the process mean and/or variability. With our proposed chart, the attempt is to enhance the performance of the classical SSEWMA chart. Numerical simulation studies indicate that the AIB-SSEWMA chart has better detection ability than the existing SSEWMA and its competitive maximum EWMA based on auxiliary information (AIB-MaxEWMA) charts in view of average run lengths (ARLs). An illustrated example is used to demonstrate the efficiency of the proposed AIB-SSEWMA chart in detecting small process shifts.