OpenRaiser/Intern-Atlas huggingface dataset:
Disentangling Identity and Pose for Facial Expression Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Facial expression recognition (FER) is a challenging problem because the expression component is always entangled with other irrelevant factors, such as identity and head pose. In this work, we propose an identity and pose disentangled facial expression recognition (IPD-FER) model to learn more discriminative feature representation. We regard the holistic facial representation as the combination of identity, pose and expression. These three components are encoded with different encoders. For identity encoder, a well pre-trained face recognition model is utilized and fixed during training, which alleviates the restriction on specific expression training data in previous works and makes the disentanglement practicable on in-the-wild datasets. At the same time, the pose and expression encoder are optimized with corresponding labels. Combining identity and pose feature, a neutral face of input individual should be generated by the decoder. When expression feature is added, the input image should be reconstructed. By comparing the difference between synthesized neutral and expressional images of the same individual, the expression component is further disentangled from identity and pose. Experimental results verify the effectiveness of our method on both lab-controlled and in-the-wild databases and we achieve state-of-the-art recognition performance.
On the Performance of Deep Learning-based Data-aided Active User Detection for GF-SCMA System
Venue: arXiv cs.IT 2022
Year: 2022

The recent works on a deep learning (DL)-based joint design of preamble set for the transmitters and data-aided active user detection (AUD) in the receiver has demonstrated a significant performance improvement for grant-free sparse code multiple access (GF-SCMA) system. The autoencoder for the joint design can be trained only in a given environment, but in an actual situation where the operating environment is constantly changing, it is difficult to optimize the preamble set for every possible environment. Therefore, a conventional, yet general approach may implement the data-aided AUD while relying on the preamble set that is designed independently rather than the joint design. In this paper, the activity detection error rate (ADER) performance of the data-aided AUD subject to the two preamble designs, i.e., independently designed preamble and jointly designed preamble, were directly compared. Fortunately, it was found that the performance loss in the data-aided AUD induced by the independent preamble design is limited to only 1dB. Furthermore, such performance characteristics of jointly designed preamble set is interpreted through average cross-correlation among the preambles associated with the same codebook (CB) (average intra-CB cross-correlation) and average cross-correlation among preambles associated with the different CBs (average inter-CB cross-correlation).
A Concept and Argumentation based Interpretable Model in High Risk Domains
Venue: arXiv cs.AI 2022
Year: 2022

Interpretability has become an essential topic for artificial intelligence in some high-risk domains such as healthcare, bank and security. For commonly-used tabular data, traditional methods trained end-to-end machine learning models with numerical and categorical data only, and did not leverage human understandable knowledge such as data descriptions. Yet mining human-level knowledge from tabular data and using it for prediction remain a challenge. Therefore, we propose a concept and argumentation based model (CAM) that includes the following two components: a novel concept mining method to obtain human understandable concepts and their relations from both descriptions of features and the underlying data, and a quantitative argumentation-based method to do knowledge representation and reasoning. As a result of it, CAM provides decisions that are based on human-level knowledge and the reasoning process is intrinsically interpretable. Finally, to visualize the purposed interpretable model, we provide a dialogical explanation that contain dominated reasoning path within CAM. Experimental results on both open source benchmark dataset and real-word business dataset show that (1) CAM is transparent and interpretable, and the knowledge inside the CAM is coherent with human understanding; (2) Our interpretable approach can reach competitive results comparing with other state-of-art models.
Robustness of the Tangle 2.0 Consensus
Venue: arXiv cs.DC 2022
Year: 2022

In this paper, we investigate the performance of the Tangle 2.0 consensus protocol in a Byzantine environment. We use an agent-based simulation model that incorporates the main features of the Tangle 2.0 consensus protocol. Our experimental results demonstrate that the Tangle 2.0 protocol is robust to the bait-and-switch attack up to the theoretical upper bound of the adversary's 33% voting weight. We further show that the common coin mechanism in Tangle 2.0 is necessary for robustness against powerful adversaries. Moreover, the experimental results confirm that the protocol can achieve around 1s confirmation time in typical scenarios and that the confirmation times of non-conflicting transactions are not affected by the presence of conflicts.
On the Elements of Datasets for Cyber Physical Systems Security
Venue: arXiv cs.CR 2022
Year: 2022

Datasets are essential to apply AI algorithms to Cyber Physical System (CPS) Security. Due to scarcity of real CPS datasets, researchers elected to generate their own datasets using either real or virtualized testbeds. However, unlike other AI domains, a CPS is a complex system with many interfaces that determine its behavior. A dataset that comprises merely a collection of sensor measurements and network traffic may not be sufficient to develop resilient AI defensive or offensive agents. In this paper, we study the \emph{elements} of CPS security datasets required to capture the system behavior and interactions, and propose a dataset architecture that has the potential to enhance the performance of AI algorithms in securing cyber physical systems. The framework includes dataset elements, attack representation, and required dataset features. We compare existing datasets to the proposed architecture to identify the current limitations and discuss the future of CPS dataset generation using testbeds.
SMPL-IK: Learned Morphology-Aware Inverse Kinematics for AI Driven Artistic Workflows
Venue: arXiv cs.GR 2022
Year: 2022

Inverse Kinematics (IK) systems are often rigid with respect to their input character, thus requiring user intervention to be adapted to new skeletons. In this paper we aim at creating a flexible, learned IK solver applicable to a wide variety of human morphologies. We extend a state-of-the-art machine learning IK solver to operate on the well known Skinned Multi-Person Linear model (SMPL). We call our model SMPL-IK, and show that when integrated into real-time 3D software, this extended system opens up opportunities for defining novel AI-assisted animation workflows. For example, pose authoring can be made more flexible with SMPL-IK by allowing users to modify gender and body shape while posing a character. Additionally, when chained with existing pose estimation algorithms, SMPL-IK accelerates posing by allowing users to bootstrap 3D scenes from 2D images while allowing for further editing. Finally, we propose a novel SMPL Shape Inversion mechanism (SMPL-SI) to map arbitrary humanoid characters to the SMPL space, allowing artists to leverage SMPL-IK on custom characters. In addition to qualitative demos showing proposed tools, we present quantitative SMPL-IK baselines on the H36M and AMASS datasets.
Discovering Agents
Venue: arXiv cs.AI 2022
Year: 2022

Causal models of agents have been used to analyse the safety aspects of machine learning systems. But identifying agents is non-trivial -- often the causal model is just assumed by the modeler without much justification -- and modelling failures can lead to mistakes in the safety analysis. This paper proposes the first formal causal definition of agents -- roughly that agents are systems that would adapt their policy if their actions influenced the world in a different way. From this we derive the first causal discovery algorithm for discovering agents from empirical data, and give algorithms for translating between causal models and game-theoretic influence diagrams. We demonstrate our approach by resolving some previous confusions caused by incorrect causal modelling of agents.
Open Long-Tailed Recognition in a Dynamic World
Venue: arXiv cs.CV 2022
Year: 2022

Real world data often exhibits a long-tailed and open-ended (with unseen classes) distribution. A practical recognition system must balance between majority (head) and minority (tail) classes, generalize across the distribution, and acknowledge novelty upon the instances of unseen classes (open classes). We define Open Long-Tailed Recognition++ (OLTR++) as learning from such naturally distributed data and optimizing for the classification accuracy over a balanced test set which includes both known and open classes. OLTR++ handles imbalanced classification, few-shot learning, open-set recognition, and active learning in one integrated algorithm, whereas existing classification approaches often focus only on one or two aspects and deliver poorly over the entire spectrum. The key challenges are: 1) how to share visual knowledge between head and tail classes, 2) how to reduce confusion between tail and open classes, and 3) how to actively explore open classes with learned knowledge. Our algorithm, OLTR++, maps images to a feature space such that visual concepts can relate to each other through a memory association mechanism and a learned metric (dynamic meta-embedding) that both respects the closed world classification of seen classes and acknowledges the novelty of open classes. Additionally, we propose an active learning scheme based on visual memory, which learns to recognize open classes in a data-efficient manner for future expansions. On three large-scale open long-tailed datasets we curated from ImageNet (object-centric), Places (scene-centric), and MS1M (face-centric) data, as well as three standard benchmarks (CIFAR-10-LT, CIFAR-100-LT, and iNaturalist-18), our approach, as a unified framework, consistently demonstrates competitive performance. Notably, our approach also shows strong potential for the active exploration of open classes and the fairness analysis of minority groups.
Extract fundamental frequency based on CNN combined with PYIN
Venue: arXiv cs.SD 2022
Year: 2022

This paper refers to the extraction of multiple fundamental frequencies (multiple F0) based on PYIN, an algorithm for extracting the fundamental frequency (F0) of monophonic music, and a trained convolutional neural networks (CNN) model, where a pitch salience function of the input signal is produced to estimate the multiple F0. The implementation of these two algorithms and their corresponding advantages and disadvantages are discussed in this article. Analysing the different performance of these two methods, PYIN is applied to supplement the F0 extracted from the trained CNN model to combine the advantages of these two algorithms. For evaluation, four pieces played by two violins are used, and the performance of the models are evaluated accoring to the flatness of the F0 curve extracted. The result shows the combined model outperforms the original algorithms when extracting F0 from monophonic music and polyphonic music.
Deep Generative Views to Mitigate Gender Classification Bias Across Gender-Race Groups
Venue: arXiv cs.CV 2022
Year: 2022

Published studies have suggested the bias of automated face-based gender classification algorithms across gender-race groups. Specifically, unequal accuracy rates were obtained for women and dark-skinned people. To mitigate the bias of gender classifiers, the vision community has developed several strategies. However, the efficacy of these mitigation strategies is demonstrated for a limited number of races mostly, Caucasian and African-American. Further, these strategies often offer a trade-off between bias and classification accuracy. To further advance the state-of-the-art, we leverage the power of generative views, structured learning, and evidential learning towards mitigating gender classification bias. We demonstrate the superiority of our bias mitigation strategy in improving classification accuracy and reducing bias across gender-racial groups through extensive experimental validation, resulting in state-of-the-art performance in intra- and cross dataset evaluations.
Auditory Feedback to Make Walking in Virtual Reality More Accessible
Venue: arXiv cs.HC 2022
Year: 2022

The objective of this study is to investigate the impact of several auditory feedback modalities on gait (i.e., walking patterns) in virtual reality (VR). Prior research has substantiated gait disturbances in VR users as one of the primary obstacles to VR usability. However, minimal research has been done to mitigate this issue. We recruited 39 participants (with mobility impairments: 18, without mobility impairments: 21) who completed timed walking tasks in a real-world environment and the same tasks in a VR environment with various types of auditory feedback. Within-subject results showed that each auditory condition significantly improved gait performance while in VR (p < .001) compared to the no auditory condition in VR for both groups of participants with and without mobility impairments. Moreover, spatial audio improved gait performance significantly (p < .001) compared to other auditory conditions for both groups of participants. This research could help to make walking in VR more accessible for people with and without mobility impairments.
Constant-Depth Sorting Networks
Venue: arXiv cs.CC 2022
Year: 2022

In this paper, we address sorting networks that are constructed from comparators of arity $k > 2$. That is, in our setting the arity of the comparators -- or, in other words, the number of inputs that can be sorted at the unit cost -- is a parameter. We study its relationship with two other parameters -- $n$, the number of inputs, and $d$, the depth. This model received considerable attention. Partly, its motivation is to better understand the structure of sorting networks. In particular, sorting networks with large arity are related to recursive constructions of ordinary sorting networks. Additionally, studies of this model have natural correspondence with a recent line of work on constructing circuits for majority functions from majority gates of lower fan-in. Motivated by these questions, we obtain the first lower bounds on the arity of constant-depth sorting networks. More precisely, we consider sorting networks of depth $d$ up to 4, and determine the minimal $k$ for which there is such a network with comparators of arity $k$. For depths $d=1,2$ we observe that $k=n$. For $d=3$ we show that $k = \lceil \frac n2 \rceil$. For $d=4$ the minimal arity becomes sublinear: $k = Θ(n^{2/3})$. This contrasts with the case of majority circuits, in which $k = O(n^{2/3})$ is achievable already for depth $d=3$.
Multi-View Correlation Consistency for Semi-Supervised Semantic Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Semi-supervised semantic segmentation needs rich and robust supervision on unlabeled data. Consistency learning enforces the same pixel to have similar features in different augmented views, which is a robust signal but neglects relationships with other pixels. In comparison, contrastive learning considers rich pairwise relationships, but it can be a conundrum to assign binary positive-negative supervision signals for pixel pairs. In this paper, we take the best of both worlds and propose multi-view correlation consistency (MVCC) learning: it considers rich pairwise relationships in self-correlation matrices and matches them across views to provide robust supervision. Together with this correlation consistency loss, we propose a view-coherent data augmentation strategy that guarantees pixel-pixel correspondence between different views. In a series of semi-supervised settings on two datasets, we report competitive accuracy compared with the state-of-the-art methods. Notably, on Cityscapes, we achieve 76.8% mIoU with 1/8 labeled data, just 0.6% shy from the fully supervised oracle.
Survey of Methods for Solving Systems of Nonlinear Equations, Part II: Optimization Based Approaches
Venue: arXiv cs.MS 2022
Year: 2022

This paper presents a comprehensive survey of methods which can be utilized to search for solutions to systems of nonlinear equations (SNEs). Our objectives with this survey are to synthesize pertinent literature in this field by presenting a thorough description and analysis of the known methods capable of finding one or many solutions to SNEs, and to assist interested readers seeking to identify solution techniques which are well suited for solving the various classes of SNEs which one may encounter in real world applications. To accomplish these objectives, we present a multi-part survey. In part one, we focused on root-finding approaches which can be used to search for solutions to a SNE without transforming it into an optimization problem. In part two, we introduce the various transformations which have been utilized to transform a SNE into an optimization problem, and we discuss optimization algorithms which can then be used to search for solutions. We emphasize the important characteristics of each method, and we discuss promising directions for future research. In part three, we will present a robust quantitative comparative analysis of methods capable of searching for solutions to SNEs.
Restructurable Activation Networks
Venue: arXiv cs.CV 2022
Year: 2022

Is it possible to restructure the non-linear activation functions in a deep network to create hardware-efficient models? To address this question, we propose a new paradigm called Restructurable Activation Networks (RANs) that manipulate the amount of non-linearity in models to improve their hardware-awareness and efficiency. First, we propose RAN-explicit (RAN-e) -- a new hardware-aware search space and a semi-automatic search algorithm -- to replace inefficient blocks with hardware-aware blocks. Next, we propose a training-free model scaling method called RAN-implicit (RAN-i) where we theoretically prove the link between network topology and its expressivity in terms of number of non-linear units. We demonstrate that our networks achieve state-of-the-art results on ImageNet at different scales and for several types of hardware. For example, compared to EfficientNet-Lite-B0, RAN-e achieves a similar accuracy while improving Frames-Per-Second (FPS) by 1.5x on Arm micro-NPUs. On the other hand, RAN-i demonstrates up to 2x reduction in #MACs over ConvNexts with a similar or better accuracy. We also show that RAN-i achieves nearly 40% higher FPS than ConvNext on Arm-based datacenter CPUs. Finally, RAN-i based object detection networks achieve a similar or higher mAP and up to 33% higher FPS on datacenter CPUs compared to ConvNext based models. The code to train and evaluate RANs and the pretrained networks are available at https://github.com/ARM-software/ML-restructurable-activation-networks.
Physical Computing for Materials Acceleration Platforms
Venue: arXiv cs.AI 2022
Year: 2022

A ''technology lottery'' describes a research idea or technology succeeding over others because it is suited to the available software and hardware, not necessarily because it is superior to alternative directions--examples abound, from the synergies of deep learning and GPUs to the disconnect of urban design and autonomous vehicles. The nascent field of Self-Driving Laboratories (SDL), particularly those implemented as Materials Acceleration Platforms (MAPs), is at risk of an analogous pitfall: the next logical step for building MAPs is to take existing lab equipment and workflows and mix in some AI and automation. In this whitepaper, we argue that the same simulation and AI tools that will accelerate the search for new materials, as part of the MAPs research program, also make possible the design of fundamentally new computing mediums. We need not be constrained by existing biases in science, mechatronics, and general-purpose computing, but rather we can pursue new vectors of engineering physics with advances in cyber-physical learning and closed-loop, self-optimizing systems. Here we outline a simulation-based MAP program to design computers that use physics itself to solve optimization problems. Such systems mitigate the hardware-software-substrate-user information losses present in every other class of MAPs and they perfect alignment between computing problems and computing mediums eliminating any technology lottery. We offer concrete steps toward early ''Physical Computing (PC) -MAP'' advances and the longer term cyber-physical R&D which we expect to introduce a new era of innovative collaboration between materials researchers and computer scientists.
GRAPHIC: GatheR-And-Process in Highly parallel with In-SSD Compression Architecture in Very Large-Scale Graph
Venue: arXiv cs.AR 2022
Year: 2022

Graph convolutional network (GCN), an emerging algorithm for graph computing, has achieved promising performance in graphstructure tasks. To achieve acceleration for data-intensive and sparse graph computing, ASICs such as GCNAX have been proposed for efficient execution of aggregation and combination in GCN. GCNAX reducing 8x DRAM accesses compared with previous efforts. However, as graphs have reached terabytes in size, off-chip data movement from SSD to DRAM becomes a serious latency bottleneck. This paper proposes Compressive Graph Transmission (CGTrans), which performs the aggregation in SSD to dramatically relieves the transfer latency bottleneck due to SSD loading compared to CMOS-based graph accelerator ASICs. InSSD computing technique is required for CGTrans. Recently, Insider was proposed as a near-SSD processing system computing by integrating FPGA in SSD. However, the Insider still suffers low area efficiency, which will limit the performance of CGTrans. The recently proposed Fully Concurrent Access Technique (FAST) is utilized. FAST-GAS, as an in-SSD graph computing accelerator, is proposed to provide high-concurrent gather-andscatter operations to overcome the area efficiency problem. We proposed the GRAPHIC system containing CGTrans dataflow deployed on FAST-GAS. Experiments show CGTrans reduces SSD loading by a factor of 50x, while GRAPHIC achieves 3.6x, and 2.4x speedup on average over GCNAX and CGTrans on Insider, respectively.
Intellectual Property Evaluation Utilizing Machine Learning
Venue: arXiv cs.AI 2022
Year: 2022

Intellectual properties is increasingly important in the economic development. To solve the pain points by traditional methods in IP evaluation, we are developing a new technology with machine learning as the core. We have built an online platform and will expand our business in the Greater Bay Area with plans.
Group-based control of large-scale micro-robot swarms with on-board Physical Finite-State Machines
Venue: arXiv cs.RO 2022
Year: 2022

An important problem in microrobotics is how to control a large group of microrobots with a global control signal. This paper focuses on controlling a large-scale swarm of MicroStressBots with on-board physical finite-state machines. We introduce the concept of group-based control, which makes it possible to scale up the swarm size while reducing the complexity both of robot fabrication as well as swarm control. We prove that the group-based control system is locally accessible in terms of the robot positions. We further hypothesize based on extensive simulations that the system is globally controllable. A nonlinear optimization strategy is proposed to control the swarm by minimizing control effort. We also propose a probabilistically complete collision avoidance method that is suitable for online use. The paper concludes with an evaluation of the proposed methods in simulations.