Miracle-dz/UniFAR-FTUs huggingface dataset:
Generalized Identifiability Bounds for Mixture Models With Grouped Samples

Recent work has shown that finite mixture models with <inline-formula> <tex-math notation="LaTeX">$m$ </tex-math></inline-formula> components are identifiable, while making no assumptions on the mixture components, so long as one has access to groups of samples of size <inline-formula> <tex-math notation="LaTeX">$2m-1$ </tex-math></inline-formula> which are known to come from the same mixture component. In this work we generalize that result and show that, if every subset of <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> mixture components of a mixture model are linearly independent, then that mixture model is identifiable with only <inline-formula> <tex-math notation="LaTeX">$(2m-1)/(k-1)$ </tex-math></inline-formula> samples per group. We further show that this value cannot be improved. We prove an analogous result for a stronger form of identifiability known as “determinedness” along with a corresponding lower bound. This independence assumption almost surely holds if mixture components are chosen randomly from a <inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>-dimensional space. We describe some implications of our results for multinomial mixture models and topic modeling.
A System-driven Automatic Ground Truth Generation Method for DL Inner-City Driving Corridor Detectors

Data-driven perception approaches are well-established in automated driving systems. In many fields even super-human performance is reached. Unlike prediction and planning approaches, mainly supervised learning algorithms are used for the perception domain. Therefore, a major remaining challenge is the efficient generation of ground truth data. As perception modules are positioned close to the sensor, they typically run on raw sensor data of high bandwidth. Due to that, the generation of ground truth labels typically causes a significant manual effort, which leads to high costs for the labelling itself and the necessary quality control. In this contribution, we propose an automatic labeling approach for semantic segmentation of the drivable ego corridor that reduces the manual effort by a factor of 150 and more. The proposed holistic approach could be used in an automated data loop, allowing a continuous improvement of the depending perception modules.
Robust Knowledge Adaptation for Dynamic Graph Neural Networks

Graph structured data often possess dynamic characters in nature, e.g., the addition of links and nodes, in many real-world applications. Recent years have witnessed the increasing attentions paid to dynamic graph neural networks for modelling such graph data, where almost all the existing approaches assume that when a new link is built, the embeddings of the neighbor nodes should be updated by learning the temporal dynamics to propagate new information. However, such approaches suffer from the limitation that if the node introduced by a new connection contains noisy information, propagating its knowledge to other nodes is not reliable and even leads to the collapse of the model. In this paper, we propose AdaNet: a robust knowledge Adaptation framework via reinforcement learning for dynamic graph neural Networks. In contrast to previous approaches immediately updating the embeddings of the neighbor nodes once adding a new link, AdaNet attempts to adaptively determine which nodes should be updated because of the new link involved. Considering that the decision whether to update the embedding of one neighbor node will have great impact on other neighbor nodes, we thus formulate the selection of node update as a sequence decision problem, and address this problem via reinforcement learning. By this means, we can adaptively propagate knowledge to other nodes for learning robust node embedding representations. To the best of our knowledge, our approach constitutes the first attempt to explore robust knowledge adaptation via reinforcement learning for dynamic graph neural networks. Extensive experiments on three benchmark datasets demonstrate that AdaNet achieves the state-of-the-art performance. In addition, we perform the experiments by adding different degrees of noise into the dataset, quantitatively and qualitatively illustrating the robustness of AdaNet.
A Machine Learning Approach for Driver Identification Based on CAN-BUS Sensor Data

: Driver identification is a momentous field of modern decorated vehicles in the perspective of the controller area network (CAN-BUS). Many conventional systems are used to identify the driver. One step ahead, most of the researchers use sensor data of CAN-BUS but there are some difficulties because of the variation of protocol of different models of vehicle. Our aim is to identify the driver through supervised learning algorithms based on driving behavior analysis. To identify the driver, a driver verification technique is proposed that evaluate driving pattern using the measurement of CAN sensor data. In this paper on-board diagnostic (OBD-II) is used to capture the data from CAN-BUS sensor and the sensors are listed under SAE J1979 statement. According to the service of OBD-II drive identification is possible. However, we have gained two types of accuracy on a full data set with 10 drivers and a partial data set with two drivers. The accuracy is good with less number of drivers compared to higher number of the driver. We have achieved statistically significant results in terms of accuracy contrast to the baseline algorithm.
Uncertainty-aware Multi-modal Learning via Cross-modal Random Network Prediction

Multi-modal learning focuses on training models by equally combining multiple input data modalities during the prediction process. However, this equal combination can be detrimental to the prediction accuracy because different modalities are usually accompanied by varying levels of uncertainty. Using such uncertainty to combine modalities has been studied by a couple of approaches, but with limited success because these approaches are either designed to deal with specific classification or segmentation problems and cannot be easily translated into other tasks, or suffer from numerical instabilities. In this paper, we propose a new Uncertainty-aware Multi-modal Learner that estimates uncertainty by measuring feature density via Cross-modal Random Network Prediction (CRNP). CRNP is designed to require little adaptation to translate between different prediction tasks, while having a stable training process. From a technical point of view, CRNP is the first approach to explore random network prediction to estimate uncertainty and to combine multi-modal data. Experiments on two 3D multi-modal medical image segmentation tasks and three 2D multi-modal computer vision classification tasks show the effectiveness, adaptability and robustness of CRNP. Also, we provide an extensive discussion on different fusion functions and visualization to validate the proposed model.
Tseng’s Algorithm with Extrapolation from the past Endowed with Variable Metrics and Error Terms

Abstract In this paper, we propose a variable metric version of Tseng’s algorithm (the forward-backward-forward algorithm: FBF) combined with extrapolation from the past that includes error terms for finding a zero of the sum of a maximally monotone operator and a monotone Lipschitzian operator in Hilbert spaces. The algorithm, which is a modified forward-reflected-backword method, is endowed with variable metrics and error terms. Primal-dual algorithms are also proposed for monotone inclusion problems involving compositions with linear operators. The primal-dual problem occurring in image deblurring demonstrates an application of our theoretical results.
Deep Sufficient Representation Learning via Mutual Information

We propose a mutual information-based sufficient representation learning (MSRL) approach, which uses the variational formulation of the mutual information and leverages the approximation power of deep neural networks. MSRL learns a sufficient representation with the maximum mutual information with the response and a user-selected distribution. It can easily handle multi-dimensional continuous or categorical response variables. MSRL is shown to be consistent in the sense that the conditional probability density function of the response variable given the learned representation converges to the conditional probability density function of the response variable given the predictor. Non-asymptotic error bounds for MSRL are also established under suitable conditions. To establish the error bounds, we derive a generalized Dudley's inequality for an order-two U-process indexed by deep neural networks, which may be of independent interest. We discuss how to determine the intrinsic dimension of the underlying data distribution. Moreover, we evaluate the performance of MSRL via extensive numerical experiments and real data analysis and demonstrate that MSRL outperforms some existing nonlinear sufficient dimension reduction methods.
Geolocated Social Media Posts are Happier: Understanding the Characteristics of Check-in Posts on Twitter

The increasing prevalence of location sharing features on social media has enabled researchers to ground computational social science research using geolocated data, affording opportunities to study human mobility, the impact of real-world events, and more. This paper analyzes what crucially separates tweets with geotags from tweets without. Our findings show that geotagged tweets are not representative of Twitter data at large, limiting the generalizability of research that uses only geolocated data. We collected 1.3M geotagged tweets on Twitter (most of which came from Instagram), and compared them with a random dataset of tweets on three aspects: affect, content, and audience engagement. We show that geotagged tweets on Twitter exhibit significantly more positivity, often citing joyous and special events such as weddings, graduations, and vacations. They also convey more collectivism by using more first-person plural pronouns and contain more additional features such as hashtags or objects in images. However, geotagged tweets generate less engagement. These findings suggest there exist significant differences in the messages conveyed in geotagged posts. Our research carries important implications for future research utilizing geolocation social media data.
There is only one realization of Lie algebras

We prove that for any reduced diﬀerential graded Lie algebra L , the classical Quillen geometrical realization h L i Q is homotopy equivalent to the realization h L i = Hom cdgl ( L • , L ) constructed via the cosimplicial free complete diﬀerential graded Lie algebra L • . As the latter is a deformation retract of the Deligne-Getzler-Hinich realization MC • ( L ) we deduce that, up to homotopy, there is only one realization functor for complete diﬀerential graded Lie algebras. Immediate consequences include an el-ementary proof of the Baues-Lemaire conjecture and the description of the Quillen realization as a representable functor.
Efficient All-Reduce for Distributed DNN Training in Optical Interconnect Systems

All-reduce is the crucial communication primitive to reduce model parameters in distributed Deep Neural Networks (DNN) training. Most existing all-reduce algorithms are designed for traditional electrical interconnect systems, which cannot meet the communication requirements for distributed training of large DNNs due to the low data bandwidth of the electrical interconnect systems. One of the promising alternatives for electrical interconnect is optical interconnect, which can provide high bandwidth, low transmission delay, and low power cost. We propose an efficient scheme called WRHT (Wavelength Reused Hierarchical Tree) for implementing all-reduce operation in optical interconnect systems. WRHT can take advantage of WDM (Wavelength Division Multiplexing) to reduce the communication time of distributed data-parallel DNN training. Simulations using real DNN models show that, compared to all-reduce algorithms in the electrical and optical network systems, our approach reduces communication time by 75.76% and 91.86%, respectively.
Fusing Frame and Event Vision for High-speed Optical Flow for Edge Application

Optical flow computation with frame-based cameras provides high accuracy but the speed is limited either by the model size of the algorithm or by the frame rate of the camera. This makes it inadequate for high-speed applications. Event cameras provide continuous asynchronous event streams overcoming the frame-rate limitation. However, the algorithms for processing the data either borrow frame like setup limiting the speed or suffer from lower accuracy. We fuse the complementary accuracy and speed advantages of the frame and event-based pipelines to provide high-speed optical flow while maintaining a low error rate. Our bio-mimetic network is validated with the MVSEC dataset showing 19% error degradation at 4$\times$ speed up. We then demonstrate the system with a high-speed drone flight scenario where a high-speed event camera computes the flow even before the optical camera sees the drone making it suited for applications like tracking and segmentation. This work shows the fundamental trade-offs in frame-based processing may be overcome by fusing data from other modalities.
RealFlow: EM-based Realistic Optical Flow Dataset Generation from Videos

Obtaining the ground truth labels from a video is challenging since the manual annotation of pixel-wise flow labels is prohibitively expensive and laborious. Besides, existing approaches try to adapt the trained model on synthetic datasets to authentic videos, which inevitably suffers from domain discrepancy and hinders the performance for real-world applications. To solve these problems, we propose RealFlow, an Expectation-Maximization based framework that can create large-scale optical flow datasets directly from any unlabeled realistic videos. Specifically, we first estimate optical flow between a pair of video frames, and then synthesize a new image from this pair based on the predicted flow. Thus the new image pairs and their corresponding flows can be regarded as a new training set. Besides, we design a Realistic Image Pair Rendering (RIPR) module that adopts softmax splatting and bi-directional hole filling techniques to alleviate the artifacts of the image synthesis. In the E-step, RIPR renders new images to create a large quantity of training data. In the M-step, we utilize the generated training data to train an optical flow network, which can be used to estimate optical flows in the next E-step. During the iterative learning steps, the capability of the flow network is gradually improved, so is the accuracy of the flow, as well as the quality of the synthesized dataset. Experimental results show that RealFlow outperforms previous dataset generation methods by a considerably large margin. Moreover, based on the generated dataset, our approach achieves state-of-the-art performance on two standard benchmarks compared with both supervised and unsupervised optical flow methods. Our code and dataset are available at https://github.com/megvii-research/RealFlow
Terahertz Communications for 6G and Beyond Wireless Networks: Challenges, Key Advancements, and Opportunities

The unprecedented increase in wireless data traffic, predicted to occur within the next decade, is motivating academia and industries to look beyond contemporary wireless standards and conceptualize the sixth-generation (6G) wireless networks. Among various promising solutions, terahertz communications (THzCom) is recognized as a highly promising technology for the 6G and beyond era, due to its unique potential to support terabit-per-second transmission in emerging applications. This article delves into key areas for developing end-to-end THzCom systems, focusing on physical, link, and network layers. Specifically, we discuss the areas of THz spectrum management, THz antennas, and beamforming, as well as the integration of other 6G-enabling technologies for THzCom. For each area, we identify the challenges imposed by the unique properties of the THz band. We then present main advancements and outline perspective research directions in each area to stimulate future research efforts for realizing THzCom in 6G and beyond wireless networks.
Positivity of higher exterior powers of the tangent bundle

We prove that a smooth projective variety X of dimension n with strictly nef third, fourth or ( n − 1)-th exterior power of the tangent bundle is a Fano variety. Moreover, in the ﬁrst two cases, we provide a classiﬁcation for X under the assumption that ρ ( X ) 6 = 1.
Adaptive Graph-Based Feature Normalization for Facial Expression Recognition

Facial Expression Recognition (FER) suffers from data uncertainties caused by ambiguous facial images and annotators' subjectiveness, resulting in excursive semantic and feature covariate shifting problem. Existing works usually correct mislabeled data by estimating noise distribution, or guide network training with knowledge learned from clean data, neglecting the associative relations of expressions. In this work, we propose an Adaptive Graph-based Feature Normalization (AGFN) method to protect FER models from data uncertainties by normalizing feature distributions with the association of expressions. Specifically, we propose a Poisson graph generator to adaptively construct topological graphs for samples in each mini-batches via a sampling process, and correspondingly design a coordinate descent strategy to optimize proposed network. Our method outperforms state-of-the-art works with accuracies of 91.84% and 91.11% on the benchmark datasets FERPlus and RAF-DB, respectively, and when the percentage of mislabeled data increases (e.g., to 20%), our network surpasses existing works significantly by 3.38% and 4.52%.
Bi-directional Contrastive Learning for Domain Adaptive Semantic Segmentation

We present a novel unsupervised domain adaptation method for semantic segmentation that generalizes a model trained with source images and corresponding ground-truth labels to a target domain. A key to domain adaptive semantic segmentation is to learn domain-invariant and discriminative features without target ground-truth labels. To this end, we propose a bi-directional pixel-prototype contrastive learning framework that minimizes intra-class variations of features for the same object class, while maximizing inter-class variations for different ones, regardless of domains. Specifically, our framework aligns pixel-level features and a prototype of the same object class in target and source images (i.e., positive pairs), respectively, sets them apart for different classes (i.e., negative pairs), and performs the alignment and separation processes toward the other direction with pixel-level features in the source image and a prototype in the target image. The cross-domain matching encourages domain-invariant feature representations, while the bidirectional pixel-prototype correspondences aggregate features for the same object class, providing discriminative features. To establish training pairs for contrastive learning, we propose to generate dynamic pseudo labels of target images using a non-parametric label transfer, that is, pixel-prototype correspondences across different domains. We also present a calibration method compensating class-wise domain biases of prototypes gradually during training.
Search-based Software Testing Driven by Automatically Generated and Manually Defined Fitness Functions

Search-based software testing (SBST) typically relies on fitness functions to guide the search exploration toward software failures. There are two main techniques to define fitness functions: (a) automated fitness function computation from the specification of the system requirements, and (b) manual fitness function design. Both techniques have advantages. The former uses information from the system requirements to guide the search toward portions of the input domain more likely to contain failures. The latter uses the engineers’ domain knowledge. We propose ATheNA, a novel SBST framework that combines fitness functions automatically generated from requirements specifications and those manually defined by engineers. We design and implement ATheNA-S, an instance of ATheNA that targets Simulink® models. We evaluate ATheNA-S by considering a large set of models from different domains. Our results show that ATheNA-S generates more failure-revealing test cases than existing baseline tools and that the difference between the runtime performance of ATheNA-S and the baseline tools is not statistically significant. We also assess whether ATheNA-S could generate failure-revealing test cases when applied to two representative case studies: one from the automotive domain and one from the medical domain. Our results show that ATheNA-S successfully revealed a requirement violation in our case studies.
Quench dynamics and scaling laws in topological nodal loop semimetals

We employ quench dynamics as an effective tool to probe different universality classes of topological phase transitions. Specifically, we study a model encompassing both Dirac-like and nodal loop criticalities. Examining the Kibble-Zurek scaling of topological defect density, we discover that the scaling exponent is reduced in the presence of extended nodal loop gap closures. For a quench through a multicritical point, we also unveil a path-dependent crossover between two sets of critical exponents. Bloch state tomography finally reveals additional differences in the defect trajectories for sudden quenches. While the Dirac transition permits a static trajectory under specific initial conditions, we find that the underlying nodal loop leads to complex time-dependent trajectories in general. In the presence of a nodal loop, we find, generically, a mismatch between the momentum modes where topological defects are generated and where dynamical quantum phase transitions occur. We also find notable exceptions where this correspondence breaks down completely.
Visual Speech-Aware Perceptual 3D Facial Expression Reconstruction from Videos

The recent state of the art on monocular 3D face reconstruction from image data has made some impressive ad-vancements, thanks to the advent of Deep Learning. However, it has mostly focused on input coming from a single RGB image, overlooking the following important factors: a) Nowadays, the vast majority of facial image data of interest do not originate from single images but rather from videos, which contain rich dynamic information. b) Furthermore, these videos typically capture individuals in some form of verbal communication (public talks, teleconferences, au-diovisual human-computer interactions, interviews, mono-logues/dialogues in movies, etc). When existing 3D face reconstruction methods are applied in such videos, the artifacts in the reconstruction of the shape and motion of the mouth area are often severe, since they do not match well with the speech audio. To overcome the aforementioned limitations, we present the ﬁrst method for visual speech-aware perceptual reconstruction of 3D for in datasets. We verify the of our method through exhaustive objective evaluations on three large-scale datasets, as well as subjective evaluation with two web-based user studies.
Quantized Sparse Weight Decomposition for Neural Network Compression

In this paper, we introduce a novel method of neural network weight compression. In our method, we store weight tensors as sparse, quantized matrix factors, whose product is computed on the fly during inference to generate the target model's weights. We use projected gradient descent methods to find quantized and sparse factorization of the weight tensors. We show that this approach can be seen as a unification of weight SVD, vector quantization, and sparse PCA. Combined with end-to-end fine-tuning our method exceeds or is on par with previous state-of-the-art methods in terms of the trade-off between accuracy and model size. Our method is applicable to both moderate compression regimes, unlike vector quantization, and extreme compression regimes.