OpenRaiser/Intern-Atlas huggingface dataset:
Rethinking Prototypical Contrastive Learning through Alignment, Uniformity and Correlation
Venue: arXiv cs.CV 2022
Year: 2022

Contrastive self-supervised learning (CSL) with a prototypical regularization has been introduced in learning meaningful representations for downstream tasks that require strong semantic information. However, to optimize CSL with a loss that performs the prototypical regularization aggressively, e.g., the ProtoNCE loss, might cause the "coagulation" of examples in the embedding space. That is, the intra-prototype diversity of samples collapses to trivial solutions for their prototype being well-separated from others. Motivated by previous works, we propose to mitigate this phenomenon by learning Prototypical representation through Alignment, Uniformity and Correlation (PAUC). Specifically, the ordinary ProtoNCE loss is revised with: (1) an alignment loss that pulls embeddings from positive prototypes together; (2) a uniformity loss that distributes the prototypical level features uniformly; (3) a correlation loss that increases the diversity and discriminability between prototypical level features. We conduct extensive experiments on various benchmarks where the results demonstrate the effectiveness of our method in improving the quality of prototypical contrastive representations. Particularly, in the classification down-stream tasks with linear probes, our proposed method outperforms the state-of-the-art instance-wise and prototypical contrastive learning methods on the ImageNet-100 dataset by 2.96% and the ImageNet-1K dataset by 2.46% under the same settings of batch size and epochs.
Prove You Owned Me: One Step beyond RFID Tag/Mutual Authentication
Venue: arXiv cs.CR 2022
Year: 2022

Radio Frequency Identification (RFID) is a key technology used in many applications. In the past decades, plenty of secure and privacy-preserving RFID tag/mutual authentication protocols as well as formal frameworks for evaluating them have been proposed. However, we notice that a property, namely proof of possession (PoP), has not been rigorously studied till now, despite it has significant value in many RFID applications. For example, in RFID-enabled supply chains, PoP helps prevent dis-honest parties from publishing information about products/tags that they actually have never processed. We propose the first formal framework for RFID tag/mutual authentication with PoP after correcting deficiencies of some existing RFID formal frameworks. We provide a generic construction to transform an RFID tag/mutual authentication protocol to one that supports PoP using a cryptographic hash function, a pseudorandom function (PRF) and a signature scheme. We prove that the constructed protocol is secure and privacy-preserving under our framework if all the building blocks possess desired security properties. Finally, we show an RFID mutual authentication protocol with PoP. Arming tag/mutual authentication protocols with PoP is an important step to strengthen RFID-enabled systems as it bridges the security gap between physical layer and data layer, and reduces the misuses of RFID-related data.
NDN-TR70 -- Utilizing NDN-DPDK for Kubernetes Genomics Data Lake
Venue: arXiv cs.NI 2022
Year: 2022

As the growth of genomics samples rapidly expands due to increased access to high resolution DNA sequencing technology, the need for a scalable platform to aggregate dispersed datasets enable easy access to the vast wealth of DNA sequences available is paramount. In this work, we introduce and demonstrate a novel way to use Named Data Networking (NDN) in conjunction with a Kubernetes cluster to design a flexible and scalable genomics Data Lake in the cloud. In addition, the use of the NDN Data Plane Development Kit (DPDK) provides an efficient and accessible distribution of the datasets to researchers anywhere. This report will explain the need to deploy a Data Lake for genomics data, what is necessary to deploy successfully, and detailed instructions to replicate the proposed design. Finally, this technical report outlines future enhancement options for further improvements.
Effective Capacity of URLLC over Parallel Fading Channels with Imperfect Channel State Information
Venue: arXiv cs.IT 2022
Year: 2022

This paper investigates the effective capacity of a point-to-point ultra-reliable low latency communication (URLLC) transmission over multiple parallel sub-channels at finite blocklength (FBL) with imperfect channel state information (CSI). Based on reasonable assumptions and approximations, we derive the effective capacity as a function of the pilot length, decoding error probability, transmit power and the sub-channel number. Then we reveal significant impact of the above parameters on the effective capacity. A closed-form lower bound of the effective capacity is derived and an alternating optimization based algorithm is proposed to find the optimal pilot length and decoding error probability. Simulation results validate our theoretical analysis and show that the closed-form lower bound is very tight. In addition, through the simulations of the optimized effective capacity, insights for pilot length and decoding error probability optimization are provided to evaluate the optimal parameters in realistic systems.
Hybrid-Regressive Neural Machine Translation
Venue: arXiv cs.CL 2022
Year: 2022

In this work, we empirically confirm that non-autoregressive translation with an iterative refinement mechanism (IR-NAT) suffers from poor acceleration robustness because it is more sensitive to decoding batch size and computing device setting than autoregressive translation (AT). Inspired by it, we attempt to investigate how to combine the strengths of autoregressive and non-autoregressive translation paradigms better. To this end, we demonstrate through synthetic experiments that prompting a small number of AT's predictions can promote one-shot non-autoregressive translation to achieve the equivalent performance of IR-NAT. Following this line, we propose a new two-stage translation prototype called hybrid-regressive translation (HRT). Specifically, HRT first generates discontinuous sequences via autoregression (e.g., make a prediction every k tokens, k>1) and then fills in all previously skipped tokens at once in a non-autoregressive manner. We also propose a bag of techniques to effectively and efficiently train HRT without adding any model parameters. HRT achieves the state-of-the-art BLEU score of 28.49 on the WMT En-De task and is at least 1.5x faster than AT, regardless of batch size and device. In addition, another bonus of HRT is that it successfully inherits the good characteristics of AT in the deep-encoder-shallow-decoder architecture. Concretely, compared to the vanilla HRT with a 6-layer encoder and 6-layer decoder, the inference speed of HRT with a 12-layer encoder and 1-layer decoder is further doubled on both GPU and CPU without BLEU loss.
Modeling the impact of external influence on green behaviour spreading in multilayer financial networks
Venue: arXiv cs.SI 2022
Year: 2022

Growing awareness of the impact of business activity on the environment increases the pressure on governing bodies to address this issue. One possibility is to encourage or force the market into green behaviours. However, it is often hard to predict how different actions affect the market. Thus, to help with that, in this paper, we have proposed the green behaviour spreading model in the bank-company multilayer network. This model allows assessing how various elements like the duration of external influence, targeted market segment, or intensity of action affect the outcome regarding market greening level. The model evaluation results indicate that governing bodies, depending on the market "openness" to green activities, can adjust the duration and intensity of the proposed action. The strength of the impact can be changed by the public or private authority with the use of obligatory or voluntary rules and the proportion of influenced banks. This research may be helpful in the process of creating the optimal setups and increasing the performance of greening policies implementation.
Multi-view Gait Recognition based on Siamese Vision Transformer
Venue: arXiv cs.CV 2022
Year: 2022

While the Vision Transformer has been used in gait recognition, its application in multi-view gait recognition is still limited. Different views significantly affect the extraction and identification accuracy of the characteristics of gait contour. To address this, this paper proposes a Siamese Mobile Vision Transformer (SMViT). This model not only focuses on the local characteristics of the human gait space but also considers the characteristics of long-distance attention associations, which can extract multi-dimensional step status characteristics. In addition, it describes how different perspectives affect gait characteristics and generate reliable perspective feature relationship factors. The average recognition rate of SMViT on the CASIA B data set reached 96.4%. The experimental results show that SMViT can attain state-of-the-art performance compared to advanced step recognition models such as GaitGAN, Multi_view GAN, Posegait and other gait recognition models.
Hierarchical Reinforcement Learning for Furniture Layout in Virtual Indoor Scenes
Venue: arXiv cs.CV 2022
Year: 2022

In real life, the decoration of 3D indoor scenes through designing furniture layout provides a rich experience for people. In this paper, we explore the furniture layout task as a Markov decision process (MDP) in virtual reality, which is solved by hierarchical reinforcement learning (HRL). The goal is to produce a proper two-furniture layout in the virtual reality of the indoor scenes. In particular, we first design a simulation environment and introduce the HRL formulation for a two-furniture layout. We then apply a hierarchical actor-critic algorithm with curriculum learning to solve the MDP. We conduct our experiments on a large-scale real-world interior layout dataset that contains industrial designs from professional designers. Our numerical results demonstrate that the proposed model yields higher-quality layouts as compared with the state-of-art models.
Robust Offline Reinforcement Learning with Gradient Penalty and Constraint Relaxation
Venue: arXiv cs.LG 2022
Year: 2022

A promising paradigm for offline reinforcement learning (RL) is to constrain the learned policy to stay close to the dataset behaviors, known as policy constraint offline RL. However, existing works heavily rely on the purity of the data, exhibiting performance degradation or even catastrophic failure when learning from contaminated datasets containing impure trajectories of diverse levels. e.g., expert level, medium level, etc., while offline contaminated data logs exist commonly in the real world. To mitigate this, we first introduce gradient penalty over the learned value function to tackle the exploding Q-functions. We then relax the closeness constraints towards non-optimal actions with critic weighted constraint relaxation. Experimental results show that the proposed techniques effectively tame the non-optimal trajectories for policy constraint offline RL methods, evaluated on a set of contaminated D4RL Mujoco and Adroit datasets.
Propagating Variational Model Uncertainty for Bioacoustic Call Label Smoothing
Venue: arXiv cs.LG 2022
Year: 2022

We focus on using the predictive uncertainty signal calculated by Bayesian neural networks to guide learning in the self-same task the model is being trained on. Not opting for costly Monte Carlo sampling of weights, we propagate the approximate hidden variance in an end-to-end manner, throughout a variational Bayesian adaptation of a ResNet with attention and squeeze-and-excitation blocks, in order to identify data samples that should contribute less into the loss value calculation. We, thus, propose uncertainty-aware, data-specific label smoothing, where the smoothing probability is dependent on this epistemic uncertainty. We show that, through the explicit usage of the epistemic uncertainty in the loss calculation, the variational model is led to improved predictive and calibration performance. This core machine learning methodology is exemplified at wildlife call detection, from audio recordings made via passive acoustic monitoring equipment in the animals' natural habitats, with the future goal of automating large scale annotation in a trustworthy manner.
Data-Augmented Counterfactual Learning for Bundle Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

Bundle Recommendation (BR) aims at recommending bundled items on online content or e-commerce platform, such as song lists on a music platform or book lists on a reading website. Several graph based models have achieved state-of-the-art performance on BR task. But their performance is still sub-optimal, since the data sparsity problem tends to be more severe in real bundle recommendation scenarios, which limits graph-based models from more sufficient learning. In this paper, we propose a novel graph learning paradigm called Counterfactual Learning for Bundle Recommendation (CLBR) to mitigate the impact of data sparsity problem and improve bundle recommendation. Our paradigm consists of two main parts: counterfactual data augmentation and counterfactual constraint. The main idea of our paradigm lies in answering the counterfactual questions: "What would a user interact with if his/her interaction history changes?" "What would a user interact with if the bundle-item affiliation relations change?" In counterfactual data augmentation, we design a heuristic sampler to generate counterfactual graph views for graph-based models, which has better noise controlling than the stochastic sampler. We further propose counterfactual loss to constrain model learning for mitigating the effects of residual noise in augmented data and achieving more sufficient model optimization. Further theoretical analysis demonstrates the rationality of our design. Extensive experiments of BR models applied with our paradigm on two real-world datasets are conducted to verify the effectiveness of the paradigm.
Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP
Venue: arXiv cs.CL 2022
Year: 2022

Textual adversarial samples play important roles in multiple subfields of NLP research, including security, evaluation, explainability, and data augmentation. However, most work mixes all these roles, obscuring the problem definitions and research goals of the security role that aims to reveal the practical concerns of NLP models. In this paper, we rethink the research paradigm of textual adversarial samples in security scenarios. We discuss the deficiencies in previous work and propose our suggestions that the research on the Security-oriented adversarial NLP (SoadNLP) should: (1) evaluate their methods on security tasks to demonstrate the real-world concerns; (2) consider real-world attackers' goals, instead of developing impractical methods. To this end, we first collect, process, and release a security datasets collection Advbench. Then, we reformalize the task and adjust the emphasis on different goals in SoadNLP. Next, we propose a simple method based on heuristic rules that can easily fulfill the actual adversarial goals to simulate real-world attack methods. We conduct experiments on both the attack and the defense sides on Advbench. Experimental results show that our method has higher practical value, indicating that the research paradigm in SoadNLP may start from our new benchmark. All the code and data of Advbench can be obtained at \url{https://github.com/thunlp/Advbench}.
Robotic Table Wiping via Reinforcement Learning and Whole-body Trajectory Optimization
Venue: arXiv cs.RO 2022
Year: 2022

We propose a framework to enable multipurpose assistive mobile robots to autonomously wipe tables to clean spills and crumbs. This problem is challenging, as it requires planning wiping actions while reasoning over uncertain latent dynamics of crumbs and spills captured via high-dimensional visual observations. Simultaneously, we must guarantee constraints satisfaction to enable safe deployment in unstructured cluttered environments. To tackle this problem, we first propose a stochastic differential equation to model crumbs and spill dynamics and absorption with a robot wiper. Using this model, we train a vision-based policy for planning wiping actions in simulation using reinforcement learning (RL). To enable zero-shot sim-to-real deployment, we dovetail the RL policy with a whole-body trajectory optimization framework to compute base and arm joint trajectories that execute the desired wiping motions while guaranteeing constraints satisfaction. We extensively validate our approach in simulation and on hardware. Video: https://youtu.be/inORKP4F3EI
An Optimization-Based Supervised Learning Algorithm for PXRD Phase Fraction Estimation
Venue: arXiv cs.LG 2022
Year: 2022

In powder diffraction data analysis, phase identification is the process of determining the crystalline phases in a sample using its characteristic Bragg peaks. For multiphasic spectra, we must also determine the relative weight fraction of each phase in the sample. Machine Learning algorithms (e.g., Artificial Neural Networks) have been applied to perform such difficult tasks in powder diffraction analysis, but typically require a significant number of training samples for acceptable performance. We have developed an approach that performs well even with a small number of training samples. We apply a fixed-point iteration algorithm on the labelled training samples to estimate monophasic spectra. Then, given an unknown sample spectrum, we again use a fixed-point iteration algorithm to determine the weighted combination of monophase spectra that best approximates the unknown sample spectrum. These weights are the desired phase fractions for the sample. We compare our approach with several traditional Machine Learning algorithms.
G-Augment: Searching for the Meta-Structure of Data Augmentation Policies for ASR
Venue: arXiv cs.LG 2022
Year: 2022

Data augmentation is a ubiquitous technique used to provide robustness to automatic speech recognition (ASR) training. However, even as so much of the ASR training process has become automated and more "end-to-end", the data augmentation policy (what augmentation functions to use, and how to apply them) remains hand-crafted. We present Graph-Augment, a technique to define the augmentation space as directed acyclic graphs (DAGs) and search over this space to optimize the augmentation policy itself. We show that given the same computational budget, policies produced by G-Augment are able to perform better than SpecAugment policies obtained by random search on fine-tuning tasks on CHiME-6 and AMI. G-Augment is also able to establish a new state-of-the-art ASR performance on the CHiME-6 evaluation set (30.7% WER). We further demonstrate that G-Augment policies show better transfer properties across warm-start to cold-start training and model size compared to random-searched SpecAugment policies.
DOT-VAE: Disentangling One Factor at a Time
Venue: arXiv cs.LG 2022
Year: 2022

As we enter the era of machine learning characterized by an overabundance of data, discovery, organization, and interpretation of the data in an unsupervised manner becomes a critical need. One promising approach to this endeavour is the problem of Disentanglement, which aims at learning the underlying generative latent factors, called the factors of variation, of the data and encoding them in disjoint latent representations. Recent advances have made efforts to solve this problem for synthetic datasets generated by a fixed set of independent factors of variation. Here, we propose to extend this to real-world datasets with a countable number of factors of variations. We propose a novel framework which augments the latent space of a Variational Autoencoders with a disentangled space and is trained using a Wake-Sleep-inspired two-step algorithm for unsupervised disentanglement. Our network learns to disentangle interpretable, independent factors from the data ``one at a time", and encode it in different dimensions of the disentangled latent space, while making no prior assumptions about the number of factors or their joint distribution. We demonstrate its quantitative and qualitative effectiveness by evaluating the latent representations learned on two synthetic benchmark datasets; DSprites and 3DShapes and on a real datasets CelebA.
Gradient Backpropagation based Feature Attribution to Enable Explainable-AI on the Edge
Venue: arXiv cs.AR 2022
Year: 2022

There has been a recent surge in the field of Explainable AI (XAI) which tackles the problem of providing insights into the behavior of black-box machine learning models. Within this field, \textit{feature attribution} encompasses methods which assign relevance scores to input features and visualize them as a heatmap. Designing flexible accelerators for multiple such algorithms is challenging since the hardware mapping of these algorithms has not been studied yet. In this work, we first analyze the dataflow of gradient backpropagation based feature attribution algorithms to determine the resource overhead required over inference. The gradient computation is optimized to minimize the memory overhead. Second, we develop a High-Level Synthesis (HLS) based configurable FPGA design that is targeted for edge devices and supports three feature attribution algorithms. Tile based computation is employed to maximally use on-chip resources while adhering to the resource constraints. Representative CNNs are trained on CIFAR-10 dataset and implemented on multiple Xilinx FPGAs using 16-bit fixed-point precision demonstrating flexibility of our library. Finally, through efficient reuse of allocated hardware resources, our design methodology demonstrates a pathway to repurpose inference accelerators to support feature attribution with minimal overhead, thereby enabling real-time XAI on the edge.
The Software Stack That Won the Formula Student Driverless Competition
Venue: arXiv cs.RO 2022
Year: 2022

This report describes our approach to design and evaluate a software stack for a race car capable of achieving competitive driving performance in the different disciplines of the Formula Student Driverless. By using a 360° LiDAR and optionally three cameras, we reliably recognize the plastic cones that mark the track boundaries at distances of around 35 m, enabling us to drive at the physical limits of the car. Using a GraphSLAM algorithm, we are able to map these cones with a root-mean-square error of less than 15 cm while driving at speeds of over 70 kph on a narrow track. The high-precision map is used in the trajectory planning to detect the lane boundaries using Delaunay triangulation and a parametric cubic spline. We calculate an optimized trajectory using a minimum curvature approach together with a GGS-diagram that takes the aerodynamics at different velocities into account. To track the target path with accelerations of up to 1.6 g, the control system is split into a PI controller for longitudinal control and model predictive controller for lateral control. Additionally, a low-level optimal control allocation is used. The software is realized in ROS C++ and tested in a custom simulation, as well as on the actual race track.