OpenRaiser/Intern-Atlas huggingface dataset:
Voice2Mesh: Cross-Modal 3D Face Model Generation from Voices
Venue: arXiv cs.GR 2021
Year: 2021

This work focuses on the analysis that whether 3D face models can be learned from only the speech inputs of speakers. Previous works for cross-modal face synthesis study image generation from voices. However, image synthesis includes variations such as hairstyles, backgrounds, and facial textures, that are arguably irrelevant to voice or without direct studies to show correlations. We instead investigate the ability to reconstruct 3D faces to concentrate on only geometry, which is more physiologically grounded. We propose both the supervised learning and unsupervised learning frameworks. Especially we demonstrate how unsupervised learning is possible in the absence of a direct voice-to-3D-face dataset under limited availability of 3D face scans when the model is equipped with knowledge distillation. To evaluate the performance, we also propose several metrics to measure the geometric fitness of two 3D faces based on points, lines, and regions. We find that 3D face shapes can be reconstructed from voices. Experimental results suggest that 3D faces can be reconstructed from voices, and our method can improve the performance over the baseline. The best performance gains (15% - 20%) on ear-to-ear distance ratio metric (ER) coincides with the intuition that one can roughly envision whether a speaker's face is overall wider or thinner only from a person's voice. See our project page for codes and data.
Federated Traffic Synthesizing and Classification Using Generative Adversarial Networks
Venue: arXiv cs.LG 2021
Year: 2021

With the fast growing demand on new services and applications as well as the increasing awareness of data protection, traditional centralized traffic classification approaches are facing unprecedented challenges. This paper introduces a novel framework, Federated Generative Adversarial Networks and Automatic Classification (FGAN-AC), which integrates decentralized data synthesizing with traffic classification. FGAN-AC is able to synthesize and classify multiple types of service data traffic from decentralized local datasets without requiring a large volume of manually labeled dataset or causing any data leakage. Two types of data synthesizing approaches have been proposed and compared: computation-efficient FGAN (FGAN-\uppercase\expandafter{\romannumeral1}) and communication-efficient FGAN (FGAN-\uppercase\expandafter{\romannumeral2}). The former only implements a single CNN model for processing each local dataset and the later only requires coordination of intermediate model training parameters. An automatic data classification and model updating framework has been proposed to automatically identify unknown traffic from the synthesized data samples and create new pseudo-labels for model training. Numerical results show that our proposed framework has the ability to synthesize highly mixed service data traffic and can significantly improve the traffic classification performance compared to existing solutions.
Model-aided Deep Reinforcement Learning for Sample-efficient UAV Trajectory Design in IoT Networks
Venue: arXiv cs.IT 2021
Year: 2021

Deep Reinforcement Learning (DRL) is gaining attention as a potential approach to design trajectories for autonomous unmanned aerial vehicles (UAV) used as flying access points in the context of cellular or Internet of Things (IoT) connectivity. DRL solutions offer the advantage of on-the-go learning hence relying on very little prior contextual information. A corresponding drawback however lies in the need for many learning episodes which severely restricts the applicability of such approach in real-world time- and energy-constrained missions. Here, we propose a model-aided deep Q-learning approach that, in contrast to previous work, considerably reduces the need for extensive training data samples, while still achieving the overarching goal of DRL, i.e to guide a battery-limited UAV on an efficient data harvesting trajectory, without prior knowledge of wireless channel characteristics and limited knowledge of wireless node locations. The key idea consists in using a small subset of nodes as anchors (i.e. with known location) and learning a model of the propagation environment while implicitly estimating the positions of regular nodes. Interaction with the model allows us to train a deep Q-network (DQN) to approximate the optimal UAV control policy. We show that in comparison with standard DRL approaches, the proposed model-aided approach requires at least one order of magnitude less training data samples to reach identical data collection performance, hence offering a first step towards making DRL a viable solution to the problem.
Unification of computer reality
Venue: arXiv cs.HC 2021
Year: 2021

The work attempts to unify the conceptual model of the user's virtual computer environment, with the aim of combining the local environments of operating systems and the global Internet environment into a single virtual environment built on general principles. To solve this problem, it is proposed to unify the conceptual basis of these environments. The existing conceptual basis of operating systems, built on the "desktop" metaphor, contains redundant concepts associated with computer architecture. The use of the spatial conceptual basis "object - place" with the concepts of "domain", "site", and "data object" allows to completely virtualize the user environment, separating it from the hardware concepts. The virtual concept "domain" is becoming a universal way of structuring the user's space. The introduction of this concept to describe the environments of operating systems provides at the mental level the integration of the structures of the local and global space. The use of the concept of "personal domain" will allow replacing the concept of "personal computer" in the mind of the user. The virtual concept of "site" as an environment for activities and data storage will allow abandoning such concepts as "application" (program), or "memory device". The site in the mind of the user is a virtual environment that includes both places for storing data objects and places for working with them. The introduction of the concept "site" into the structure of operating systems environments and the concept of "data site" into the structure of the global network integrates the structure of the global and local space in the user's mind. The introduction of the concept of "portal" as a means of integrating information necessary for interaction, allows ensuring the methodological homogeneity of the user's work in a single virtual environment.
Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?
Venue: arXiv cs.CL 2021
Year: 2021

Most work in NLP makes the assumption that it is desirable to develop solutions in the native language in question. There is consequently a strong trend towards building native language models even for low-resource languages. This paper questions this development, and explores the idea of simply translating the data into English, thereby enabling the use of pretrained, and large-scale, English language models. We demonstrate empirically that a large English language model coupled with modern machine translation outperforms native language models in most Scandinavian languages. The exception to this is Finnish, which we assume is due to inferior translation quality. Our results suggest that machine translation is a mature technology, which raises a serious counter-argument for training native language models for low-resource languages. This paper therefore strives to make a provocative but important point. As English language models are improving at an unprecedented pace, which in turn improves machine translation, it is from an empirical and environmental stand-point more effective to translate data from low-resource languages into English, than to build language models for such languages.
Making Differentiable Architecture Search less local
Venue: arXiv cs.LG 2021
Year: 2021

Neural architecture search (NAS) is a recent methodology for automating the design of neural network architectures. Differentiable neural architecture search (DARTS) is a promising NAS approach that dramatically increases search efficiency. However, it has been shown to suffer from performance collapse, where the search often leads to detrimental architectures. Many recent works try to address this issue of DARTS by identifying indicators for early stopping, regularising the search objective to reduce the dominance of some operations, or changing the parameterisation of the search problem. In this work, we hypothesise that performance collapses can arise from poor local optima around typical initial architectures and weights. We address this issue by developing a more global optimisation scheme that is able to better explore the space without changing the DARTS problem formulation. Our experiments show that our changes in the search algorithm allow the discovery of architectures with both better test performance and fewer parameters.
Interpretation of multi-label classification models using shapley values
Venue: arXiv cs.LG 2021
Year: 2021

Multi-label classification is a type of classification task, it is used when there are two or more classes, and the data point we want to predict may belong to none of the classes or all of them at the same time. In the real world, many applications are actually multi-label involved, including information retrieval, multimedia content annotation, web mining, and so on. A game theory-based framework known as SHapley Additive exPlanations (SHAP) has been applied to explain various supervised learning models without being aware of the exact model. Herein, this work further extends the explanation of multi-label classification task by using the SHAP methodology. The experiment demonstrates a comprehensive comparision of different algorithms on well known multi-label datasets and shows the usefulness of the interpretation.
Searching with Opponent-Awareness
Venue: arXiv cs.MA 2021
Year: 2021

We propose Searching with Opponent-Awareness (SOA), an approach to leverage opponent-aware planning without explicit or a priori opponent models for improving performance and social welfare in multi-agent systems. To this end, we develop an opponent-aware MCTS scheme using multi-armed bandits based on Learning with Opponent-Learning Awareness (LOLA) and compare its effectiveness with other bandits, including UCB1. Our evaluations include several different settings and show the benefits of SOA are especially evident with increasing number of agents.
Balanced Knowledge Distillation for Long-tailed Learning
Venue: arXiv cs.CV 2021
Year: 2021

Deep models trained on long-tailed datasets exhibit unsatisfactory performance on tail classes. Existing methods usually modify the classification loss to increase the learning focus on tail classes, which unexpectedly sacrifice the performance on head classes. In fact, this scheme leads to a contradiction between the two goals of long-tailed learning, i.e., learning generalizable representations and facilitating learning for tail classes. In this work, we explore knowledge distillation in long-tailed scenarios and propose a novel distillation framework, named Balanced Knowledge Distillation (BKD), to disentangle the contradiction between the two goals and achieve both simultaneously. Specifically, given a vanilla teacher model, we train the student model by minimizing the combination of an instance-balanced classification loss and a class-balanced distillation loss. The former benefits from the sample diversity and learns generalizable representation, while the latter considers the class priors and facilitates learning mainly for tail classes. The student model trained with BKD obtains significant performance gain even compared with its teacher model. We conduct extensive experiments on several long-tailed benchmark datasets and demonstrate that the proposed BKD is an effective knowledge distillation framework in long-tailed scenarios, as well as a new state-of-the-art method for long-tailed learning. Code is available at https://github.com/EricZsy/BalancedKnowledgeDistillation .
Random perfect information games
Venue: arXiv cs.GT 2021
Year: 2021

The paper proposes a natural measure space of zero-sum perfect information games with upper semicontinuous payoffs. Each game is specified by the game tree, and by the assignment of the active player and of the capacity to each node of the tree. The payoff in a game is defined as the infimum of the capacity over the nodes that have been visited during the play. The active player, the number of children, and the capacity are drawn from a given joint distribution independently across the nodes. We characterize the cumulative distribution function of the value $v$ using the fixed points of the so-called value generating function. The characterization leads to a necessary and sufficient condition for the event $v \geq k$ to occur with positive probability. We also study probabilistic properties of the set of Player I's $k$-optimal strategies and the corresponding plays.
A Lightweight Concept Drift Detection and Adaptation Framework for IoT Data Streams
Venue: arXiv cs.LG 2021
Year: 2021

In recent years, with the increasing popularity of "Smart Technology", the number of Internet of Things (IoT) devices and systems have surged significantly. Various IoT services and functionalities are based on the analytics of IoT streaming data. However, IoT data analytics faces concept drift challenges due to the dynamic nature of IoT systems and the ever-changing patterns of IoT data streams. In this article, we propose an adaptive IoT streaming data analytics framework for anomaly detection use cases based on optimized LightGBM and concept drift adaptation. A novel drift adaptation method named Optimized Adaptive and Sliding Windowing (OASW) is proposed to adapt to the pattern changes of online IoT data streams. Experiments on two public datasets show the high accuracy and efficiency of our proposed adaptive LightGBM model compared against other state-of-the-art approaches. The proposed adaptive LightGBM model can perform continuous learning and drift adaptation on IoT data streams without human intervention.
How to Identify and Authenticate Users in Massive Unsourced Random Access
Venue: arXiv cs.IT 2021
Year: 2021

Identification and authentication are two basic functionalities of traditional random access protocols. In ALOHA-based random access, the packets usually include a field with a unique user address. However, when the number of users is massive and relatively small packets are transmitted, the overhead of including such field becomes restrictive. In unsourced random access (U-RA), the packets do not include any address field for the user, which maximizes the number of useful bits that are transmitted. However, by definition an U-RA protocol does not provide user identification. This paper presents a scheme that builds upon an underlying U-RA protocol and solves the problem of user identification and authentication. In our scheme, the users generate a message authentication code (MAC) that provides these functionalities without violating the main principle of unsourced random access: the selection of codewords from a common codebook is i.i.d. among all users.
The Orbit Problem for Parametric Linear Dynamical Systems
Venue: arXiv cs.LO 2021
Year: 2021

We study a parametric version of the Kannan-Lipton Orbit Problem for linear dynamical systems. We show decidability in the case of one parameter and Skolem-hardness with two or more parameters. More precisely, consider a $d$-dimensional square matrix $M$ whose entries are algebraic functions in one or more real variables. Given initial and target vectors $u,v\in \mathbb{Q}^d$, the parametric point-to-point orbit problem asks whether there exist values of the parameters giving rise to a concrete matrix $N \in \mathbb{R}^{d\times d}$, and a positive integer $n\in \mathbb{N}$, such that $N^nu = v$. We show decidability for the case in which $M$ depends only upon a single parameter, and we exhibit a reduction from the well-known Skolem Problem for linear recurrence sequences, suggesting intractability in the case of two or more parameters.
Exerting Control in Repeated Social Dilemmas with Thresholds
Venue: arXiv cs.MA 2021
Year: 2021

Situations in which immediate self-interest and long-term collective interest conflict often require some form of influence to prevent them from leading to undesirable or unsustainable outcomes. Next to sanctioning, social influence and social structure, it is possible that strategic solutions can exist for these social dilemmas. However, the existence of strategies that enable a player to exert control in the long-run outcomes can be difficult to show and different situations allow for different levels of strategic influence. Here, we investigate the effect of threshold nonlinearities on the possibilities of exerting unilateral control in finitely repeated n-player public goods games and snowdrift games. These models can describe situations in which a collective effort is necessary in order for a benefit to be created. We identify conditions in terms of a cooperator threshold for the existence of generous, extortionate and equalizing zero-determinant (ZD) strategies. Our results show that, for both games, the thresholds prevent equalizing ZD strategies from existing. In the snowdrift game, introducing a cooperator threshold has no effect on the region of feasible extortionate ZD strategies. For extortionate strategies in the public goods game, the threshold only restricts the region of enforceable strategies for small values of the public goods multiplier. Generous ZD strategies exist for both games, but introducing a cooperator threshold forces the slope more towards the value of a fair strategy, where the player has approximately the same payoff as the average payoff of his opponents.
TransICD: Transformer Based Code-wise Attention Model for Explainable ICD Coding
Venue: arXiv cs.CL 2021
Year: 2021

International Classification of Disease (ICD) coding procedure which refers to tagging medical notes with diagnosis codes has been shown to be effective and crucial to the billing system in medical sector. Currently, ICD codes are assigned to a clinical note manually which is likely to cause many errors. Moreover, training skilled coders also requires time and human resources. Therefore, automating the ICD code determination process is an important task. With the advancement of artificial intelligence theory and computational hardware, machine learning approach has emerged as a suitable solution to automate this process. In this project, we apply a transformer-based architecture to capture the interdependence among the tokens of a document and then use a code-wise attention mechanism to learn code-specific representations of the entire document. Finally, they are fed to separate dense layers for corresponding code prediction. Furthermore, to handle the imbalance in the code frequency of clinical datasets, we employ a label distribution aware margin (LDAM) loss function. The experimental results on the MIMIC-III dataset show that our proposed model outperforms other baselines by a significant margin. In particular, our best setting achieves a micro-AUC score of 0.923 compared to 0.868 of bidirectional recurrent neural networks. We also show that by using the code-wise attention mechanism, the model can provide more insights about its prediction, and thus it can support clinicians to make reliable decisions. Our code is available online (https://github.com/biplob1ly/TransICD)
Explainable artificial intelligence for mechanics: physics-informing neural networks for constitutive models
Venue: arXiv cs.LG 2021
Year: 2021

(Artificial) neural networks have become increasingly popular in mechanics to accelerate computations with model order reduction techniques and as universal models for a wide variety of materials. However, the major disadvantage of neural networks remains: their numerous parameters are challenging to interpret and explain. Thus, neural networks are often labeled as black boxes, and their results often elude human interpretation. In mechanics, the new and active field of physics-informed neural networks attempts to mitigate this disadvantage by designing deep neural networks on the basis of mechanical knowledge. By using this a priori knowledge, deeper and more complex neural networks became feasible, since the mechanical assumptions could be explained. However, the internal reasoning and explanation of neural network parameters remain mysterious. Complementary to the physics-informed approach, we propose a first step towards a physics-informing approach, which explains neural networks trained on mechanical data a posteriori. This novel explainable artificial intelligence approach aims at elucidating the black box of neural networks and their high-dimensional representations. Therein, the principal component analysis decorrelates the distributed representations in cell states of RNNs and allows the comparison to known and fundamental functions. The novel approach is supported by a systematic hyperparameter search strategy that identifies the best neural network architectures and training parameters. The findings of three case studies on fundamental constitutive models (hyperelasticity, elastoplasticity, and viscoelasticity) imply that the proposed strategy can help identify numerical and analytical closed-form solutions to characterize new materials.
Deep limits and cut-off phenomena for neural networks
Venue: arXiv cs.LG 2021
Year: 2021

We consider dynamical and geometrical aspects of deep learning. For many standard choices of layer maps we display semi-invariant metrics which quantify differences between data or decision functions. This allows us, when considering random layer maps and using non-commutative ergodic theorems, to deduce that certain limits exist when letting the number of layers tend to infinity. We also examine the random initialization of standard networks where we observe a surprising cut-off phenomenon in terms of the number of layers, the depth of the network. This could be a relevant parameter when choosing an appropriate number of layers for a given learning task, or for selecting a good initialization procedure. More generally, we hope that the notions and results in this paper can provide a framework, in particular a geometric one, for a part of the theoretical understanding of deep neural networks.
AdaptiFont: Increasing Individuals' Reading Speed with a Generative Font Model and Bayesian Optimization
Venue: arXiv cs.HC 2021
Year: 2021

Digital text has become one of the primary ways of exchanging knowledge, but text needs to be rendered to a screen to be read. We present AdaptiFont, a human-in-the-loop system that is aimed at interactively increasing readability of text displayed on a monitor. To this end, we first learn a generative font space with non-negative matrix factorization from a set of classic fonts. In this space we generate new true-type-fonts through active learning, render texts with the new font, and measure individual users' reading speed. Bayesian optimization sequentially generates new fonts on the fly to progressively increase individuals' reading speed. The results of a user study show that this adaptive font generation system finds regions in the font space corresponding to high reading speeds, that these fonts significantly increase participants' reading speed, and that the found fonts are significantly different across individual readers.