OpenRaiser/Intern-Atlas huggingface dataset:
Conditional Generative Data-free Knowledge Distillation
Venue: arXiv cs.CV 2022
Year: 2022

Knowledge distillation has made remarkable achievements in model compression. However, most existing methods require the original training data, which is usually unavailable due to privacy and security issues. In this paper, we propose a conditional generative data-free knowledge distillation (CGDD) framework for training lightweight networks without any training data. This method realizes efficient knowledge distillation based on conditional image generation. Specifically, we treat the preset labels as ground truth to train a conditional generator in a semi-supervised manner. The trained generator can produce specified classes of training images. For training the student network, we force it to extract the knowledge hidden in teacher feature maps, which provide crucial cues for the learning process. Moreover, an adversarial training framework for promoting distillation performance is constructed by designing several loss functions. This framework helps the student model to explore larger data space. To demonstrate the effectiveness of the proposed method, we conduct extensive experiments on different datasets. Compared with other data-free works, our work obtains state-of-the-art results on CIFAR100, Caltech101, and different versions of ImageNet datasets. The codes will be released.
Hypers at ComMA@ICON: Modelling Aggressiveness, Gender Bias and Communal Bias Identification
Venue: arXiv cs.CL 2022
Year: 2022

Due to the exponentially increasing reach of social media, it is essential to focus on its negative aspects as it can potentially divide society and incite people into violence. In this paper, we present our system description of work on the shared task ComMA@ICON, where we have to classify how aggressive the sentence is and if the sentence is gender-biased or communal biased. These three could be the primary reasons to cause significant problems in society. As team Hypers we have proposed an approach that utilizes different pretrained models with Attention and mean pooling methods. We were able to get Rank 3 with 0.223 Instance F1 score on Bengali, Rank 2 with 0.322 Instance F1 score on Multi-lingual set, Rank 4 with 0.129 Instance F1 score on Meitei and Rank 5 with 0.336 Instance F1 score on Hindi. The source code and the pretrained models of this work can be found here.
Accurate and Real-time 3D Pedestrian Detection Using an Efficient Attentive Pillar Network
Venue: arXiv cs.CV 2022
Year: 2022

Efficiently and accurately detecting people from 3D point cloud data is of great importance in many robotic and autonomous driving applications. This fundamental perception task is still very challenging due to (i) significant deformations of human body pose and gesture over time and (ii) point cloud sparsity and scarcity for pedestrian class objects. Recent efficient 3D object detection approaches rely on pillar features to detect objects from point cloud data. However, these pillar features do not carry sufficient expressive representations to deal with all the aforementioned challenges in detecting people. To address this shortcoming, we first introduce a stackable Pillar Aware Attention (PAA) module for enhanced pillar features extraction while suppressing noises in the point clouds. By integrating multi-point-channel-pooling, point-wise, channel-wise, and task-aware attention into a simple module, the representation capabilities are boosted while requiring little additional computing resources. We also present Mini-BiFPN, a small yet effective feature network that creates bidirectional information flow and multi-level cross-scale feature fusion to better integrate multi-resolution features. Our proposed framework, namely PiFeNet, has been evaluated on three popular large-scale datasets for 3D pedestrian Detection, i.e. KITTI, JRDB, and nuScenes achieving state-of-the-art (SOTA) performance on KITTI Bird-eye-view (BEV) and JRDB and very competitive performance on nuScenes. Our approach has inference speed of 26 frame-per-second (FPS), making it a real-time detector. The code for our PiFeNet is available at https://github.com/ldtho/PiFeNet.
Polynomial-Time Key Recovery Attack on the Lau-Tan Cryptosystem Based on Gabidulin Codes
Venue: arXiv cs.IT 2022
Year: 2022

This paper presents a key recovery attack on the cryptosystem proposed by Lau and Tan in a talk at ACISP 2018. The Lau-Tan cryptosystem uses Gabidulin codes as the underlying decodable code. To hide the algebraic structure of Gabidulin codes, the authors chose a matrix of column rank $n$ to mix with a generator matrix of the secret Gabidulin code. The other part of the public key, however, reveals crucial information about the private key. Our analysis shows that the problem of recovering the private key can be reduced to solving a multivariate linear system over the base field, rather than solving a multivariate quadratic system as claimed by the authors. Solving the linear system for any nonzero solution permits us to recover the private key. Apparently, this attack costs polynomial time, and therefore completely breaks the cryptosystem.
Information Systems Dynamics: Foundations and Applications
Venue: arXiv cs.IT 2022
Year: 2022

This article firstly reviews and summarizes the rapid development of information technology, characterized by the close combination of computer and network communication, which leads to a series of investigations, including the analyses of the important role of a series of technological achievements in the context of information movement and application, the interrelationship between the real-world, information space and information system, and the integrated framework of the real-world and information system, and the modifications and improvements of the Xu's previous mathematical theory on information models, properties and metrics. Based on the mathematical foundations, eleven types of information measure efficacies and their distribution across information systems are put forward, and then the dynamics configurations of information systems are comprehensively analyzed, which constitutes the basic theoretical framework of information systems dynamics with general significance. Finally, Smart Court SoSs (System of Systems) Engineering Project of China are introduced as the exemplified application of the theoretical work, which aims at providing a reference for the analysis, design, development and evaluation of large-scale complex information systems.
TransLog: A Unified Transformer-based Framework for Log Anomaly Detection
Venue: arXiv cs.LG 2022
Year: 2022

Log anomaly detection is a key component in the field of artificial intelligence for IT operations (AIOps). Considering log data of variant domains, retraining the whole network for unknown domains is inefficient in real industrial scenarios especially for low-resource domains. However, previous deep models merely focused on extracting the semantics of log sequence in the same domain, leading to poor generalization on multi-domain logs. Therefore, we propose a unified Transformer-based framework for log anomaly detection (\ourmethod{}), which is comprised of the pretraining and adapter-based tuning stage. Our model is first pretrained on the source domain to obtain shared semantic knowledge of log data. Then, we transfer the pretrained model to the target domain via the adapter-based tuning. The proposed method is evaluated on three public datasets including one source domain and two target domains. The experimental results demonstrate that our simple yet efficient approach, with fewer trainable parameters and lower training costs in the target domain, achieves state-of-the-art performance on three benchmarks.
Role of Data Augmentation Strategies in Knowledge Distillation for Wearable Sensor Data
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks are parametrized by several thousands or millions of parameters, and have shown tremendous success in many classification problems. However, the large number of parameters makes it difficult to integrate these models into edge devices such as smartphones and wearable devices. To address this problem, knowledge distillation (KD) has been widely employed, that uses a pre-trained high capacity network to train a much smaller network, suitable for edge devices. In this paper, for the first time, we study the applicability and challenges of using KD for time-series data for wearable devices. Successful application of KD requires specific choices of data augmentation methods during training. However, it is not yet known if there exists a coherent strategy for choosing an augmentation approach during KD. In this paper, we report the results of a detailed study that compares and contrasts various common choices and some hybrid data augmentation strategies in KD based human activity analysis. Research in this area is often limited as there are not many comprehensive databases available in the public domain from wearable devices. Our study considers databases from small scale publicly available to one derived from a large scale interventional study into human activity and sedentary behavior. We find that the choice of data augmentation techniques during KD have a variable level of impact on end performance, and find that the optimal network choice as well as data augmentation strategies are specific to a dataset at hand. However, we also conclude with a general set of recommendations that can provide a strong baseline performance across databases.
Usability and Aesthetics: Better Together for Automated Repair of Web Pages
Venue: arXiv cs.SE 2022
Year: 2022

With the recent explosive growth of mobile devices such as smartphones or tablets, guaranteeing consistent web appearance across all environments has become a significant problem. This happens simply because it is hard to keep track of the web appearance on different sizes and types of devices that render the web pages. Therefore, fixing the inconsistent appearance of web pages can be difficult, and the cost incurred can be huge, e.g., poor user experience and financial loss due to it. Recently, automated web repair techniques have been proposed to automatically resolve inconsistent web page appearance, focusing on improving usability. However, generated patches tend to disrupt the webpage's layout, rendering the repaired webpage aesthetically unpleasing, e.g., distorted images or misalignment of components. In this paper, we propose an automated repair approach for web pages based on meta-heuristic algorithms that can assure both usability and aesthetics. The key novelty that empowers our approach is a novel fitness function that allows us to optimistically evolve buggy web pages to find the best solution that optimizes both usability and aesthetics at the same time. Empirical evaluations show that our approach is able to successfully resolve mobile-friendly problems in 94% of the evaluation subjects, significantly outperforming state-of-the-art baseline techniques in terms of both usability and aesthetics.
A Surrogate-Assisted Controller for Expensive Evolutionary Reinforcement Learning
Venue: arXiv cs.NE 2022
Year: 2022

The integration of Reinforcement Learning (RL) and Evolutionary Algorithms (EAs) aims at simultaneously exploiting the sample efficiency as well as the diversity and robustness of the two paradigms. Recently, hybrid learning frameworks based on this principle have achieved great success in various challenging robot control tasks. However, in these methods, policies from the genetic population are evaluated via interactions with the real environments, limiting their applicability in computationally expensive problems. In this work, we propose Surrogate-assisted Controller (SC), a novel and efficient module that can be integrated into existing frameworks to alleviate the computational burden of EAs by partially replacing the expensive policy evaluation. The key challenge in applying this module is to prevent the optimization process from being misled by the possible false minima introduced by the surrogate. To address this issue, we present two strategies for SC to control the workflow of hybrid frameworks. Experiments on six continuous control tasks from the OpenAI Gym platform show that SC can not only significantly reduce the cost of fitness evaluations, but also boost the performance of the original hybrid frameworks with collaborative learning and evolutionary processes.
Toward Pareto Efficient Fairness-Utility Trade-off inRecommendation through Reinforcement Learning
Venue: arXiv cs.IR 2022
Year: 2022

The issue of fairness in recommendation is becoming increasingly essential as Recommender Systems touch and influence more and more people in their daily lives. In fairness-aware recommendation, most of the existing algorithmic approaches mainly aim at solving a constrained optimization problem by imposing a constraint on the level of fairness while optimizing the main recommendation objective, e.g., CTR. While this alleviates the impact of unfair recommendations, the expected return of an approach may significantly compromise the recommendation accuracy due to the inherent trade-off between fairness and utility. This motivates us to deal with these conflicting objectives and explore the optimal trade-off between them in recommendation. One conspicuous approach is to seek a Pareto efficient solution to guarantee optimal compromises between utility and fairness. Moreover, considering the needs of real-world e-commerce platforms, it would be more desirable if we can generalize the whole Pareto Frontier, so that the decision-makers can specify any preference of one objective over another based on their current business needs. Therefore, in this work, we propose a fairness-aware recommendation framework using multi-objective reinforcement learning, called MoFIR, which is able to learn a single parametric representation for optimal recommendation policies over the space of all possible preferences. Specially, we modify traditional DDPG by introducing conditioned network into it, which conditions the networks directly on these preferences and outputs Q-value-vectors. Experiments on several real-world recommendation datasets verify the superiority of our framework on both fairness metrics and recommendation measures when compared with all other baselines. We also extract the approximate Pareto Frontier on real-world datasets generated by MoFIR and compare to state-of-the-art fairness methods.
High-dimensional Bayesian Optimization Algorithm with Recurrent Neural Network for Disease Control Models in Time Series
Venue: arXiv cs.LG 2022
Year: 2022

Bayesian Optimization algorithm has become a promising approach for nonlinear global optimization problems and many machine learning applications. Over the past few years, improvements and enhancements have been brought forward and they have shown some promising results in solving the complex dynamic problems, systems of ordinary differential equations where the objective functions are computationally expensive to evaluate. Besides, the straightforward implementation of the Bayesian Optimization algorithm performs well merely for optimization problems with 10-20 dimensions. The study presented in this paper proposes a new high dimensional Bayesian Optimization algorithm combining Recurrent neural networks, which is expected to predict the optimal solution for the global optimization problems with high dimensional or time series decision models. The proposed RNN-BO algorithm can solve the optimal control problems in the lower dimension space and then learn from the historical data using the recurrent neural network to learn the historical optimal solution data and predict the optimal control strategy for any new initial system value setting. In addition, accurately and quickly providing the optimal control strategy is essential to effectively and efficiently control the epidemic spread while minimizing the associated financial costs. Therefore, to verify the effectiveness of the proposed algorithm, computational experiments are carried out on a deterministic SEIR epidemic model and a stochastic SIS optimal control model. Finally, we also discuss the impacts of different numbers of the RNN layers and training epochs on the trade-off between solution quality and related computational efforts.
Subfield prestige and gender inequality in computing
Venue: arXiv cs.CY 2022
Year: 2022

Women and people of color remain dramatically underrepresented among computing faculty, and improvements in demographic diversity are slow and uneven. Effective diversification strategies depend on quantifying the correlates, causes, and trends of diversity in the field. But field-level demographic changes are driven by subfield hiring dynamics because faculty searches are typically at the subfield level. Here, we quantify and forecast variations in the demographic composition of the subfields of computing using a comprehensive database of training and employment records for 6882 tenure-track faculty from 269 PhD-granting computing departments in the United States, linked with 327,969 publications. We find that subfield prestige correlates with gender inequality, such that faculty working in computing subfields with more women tend to hold positions at less prestigious institutions. In contrast, we find no significant evidence of racial or socioeconomic differences by subfield. Tracking representation over time, we find steady progress toward gender equality in all subfields, but more prestigious subfields tend to be roughly 25 years behind the less prestigious subfields in gender representation. These results illustrate how the choice of subfield in a faculty search can shape a department's gender diversity.
The solutions to single-variable polynomials, implemented and verified in Lean
Venue: arXiv cs.LO 2022
Year: 2022

In this work, we describe our experience in learning the use of a computer proof assistant - specifically, Lean - from scratch, through proving formulae for the solutions of polynomial equations. Specifically, in this work we characterize the solutions of quadratic, cubic, and quartic polynomials over certain fields, specifically, fields with operations returning square and cubic roots of characteristic other than two or three. The purpose of this work is thus twofold. Firstly, it describes the learning experience of a starting Lean user, including a detailed comparison between our work in Lean and very closely related work in Coq. Secondly, our results represent a modest improvement over the aforementioned related work in Coq, which we hope will be of some scientific interest.
Reinforcement Learning for Task Specifications with Action-Constraints
Venue: arXiv cs.LG 2022
Year: 2022

In this paper, we use concepts from supervisory control theory of discrete event systems to propose a method to learn optimal control policies for a finite-state Markov Decision Process (MDP) in which (only) certain sequences of actions are deemed unsafe (respectively safe). We assume that the set of action sequences that are deemed unsafe and/or safe are given in terms of a finite-state automaton; and propose a supervisor that disables a subset of actions at every state of the MDP so that the constraints on action sequence are satisfied. Then we present a version of the Q-learning algorithm for learning optimal policies in the presence of non-Markovian action-sequence and state constraints, where we use the development of reward machines to handle the state constraints. We illustrate the method using an example that captures the utility of automata-based methods for non-Markovian state and action specifications for reinforcement learning and show the results of simulations in this setting.
V-LinkNet: Learning Contextual Inpainting Across Latent Space of Generative Adversarial Network
Venue: arXiv cs.CV 2022
Year: 2022

Image inpainting is a key technique in image processing task to predict the missing regions and generate realistic images. Given the advancement of existing generative inpainting models with feature extraction, propagation and reconstruction capabilities, there is lack of high-quality feature extraction and transfer mechanisms in deeper layers to tackle persistent aberrations on the generated inpainted regions. Our method, V-LinkNet, develops high-level feature transference to deep level textural context of inpainted regions our work, proposes a novel technique of combining encoders learning through a recursive residual transition layer (RSTL). The RSTL layer easily adapts dual encoders by increasing the unique semantic information through direct communication. By collaborating the dual encoders structure with contextualised feature representation loss function, our system gains the ability to inpaint with high-level features. To reduce biases from random mask-image pairing, we introduce a standard protocol with paired mask-image on the testing set of CelebA-HQ, Paris Street View and Places2 datasets. Our results show V-LinkNet performed better on CelebA-HQ and Paris Street View using this standard protocol. We will share the standard protocol and our codes with the research community upon acceptance of this paper.
A Systematic Literature Review on Persuasive Technology at the Workplace
Venue: arXiv cs.HC 2022
Year: 2022

Employees face decisions every day - in the absence of supervision. The outcome of these decisions can be influenced by digital workplace design through the power of persuasive technology. This paper provides a structured literature review based on recent research on persuasive technology in the workplace. It examines the design and use of persuasive systems from a variety of disciplinary perspectives and theories. The reviewed studies were categorized into the research streams of technology design, user-centered research, and gamification. The purpose of the studies is categorized using a modified definition of the persuasive systems design model. A number of experimental studies show that alignment of the employee's behavior with the employer's agenda can be achieved. A robust finding is the key role of interactivity in granting employees a subjective experience of rapid and meaningful feedback when using the interface.
Visilence: An Interactive Visualization Tool for Error Resilience Analysis
Venue: arXiv cs.HC 2022
Year: 2022

Soft errors have become one of the major concerns for HPC applications, as those errors can result in seriously corrupted outcomes, such as silent data corruptions (SDCs). Prior studies on error resilience have studied the robustness of HPC applications. However, it is still difficult for program developers to identify potential vulnerability to soft errors. In this paper, we present Visilence, a novel visualization tool to visually analyze error vulnerability based on the control-flow graph generated from HPC applications. Visilence efficiently visualizes the affected program states under injected errors and presents the visual analysis of the most vulnerable parts of an application. We demonstrate the effectiveness of Visilence through a case study.