Reviewerly/exHarmony huggingface dataset:
SSFG: Stochastically Scaling Features and Gradients for Regularizing Graph Convolutional Networks

Authors:
raw_author_name: Zhang, Haimin
raw_affiliation_string: NTT Media Intelligence Laboratories, NTT Cor-poration, Chiyoda-ku 100-8116, Japan; School of Electrical and Data Engineering, Faculty of Engi-neering and Information Technology, University of Technology Sydney, 15 Broadway, Ultimo, NSW 2007, Australia
raw_author_name: Xu, Min
raw_affiliation_string: NTT Media Intelligence Laboratories, NTT Cor-poration, Chiyoda-ku 100-8116, Japan; School of Electrical and Data Engineering, Faculty of Engi-neering and Information Technology, University of Technology Sydney, 15 Broadway, Ultimo, NSW 2007, Australia
raw_author_name: Zhang, Guoqiang
raw_affiliation_string: NTT Media Intelligence Laboratories, NTT Cor-poration, Chiyoda-ku 100-8116, Japan; School of Electrical and Data Engineering, Faculty of Engi-neering and Information Technology, University of Technology Sydney, 15 Broadway, Ultimo, NSW 2007, Australia
raw_author_name: Niwa, Kenta
raw_affiliation_string: NTT Media Intelligence Laboratories, NTT Cor-poration, Chiyoda-ku 100-8116, Japan; School of Electrical and Data Engineering, Faculty of Engi-neering and Information Technology, University of Technology Sydney, 15 Broadway, Ultimo, NSW 2007, Australia

Abstract: Graph convolutional networks (GCNs) have been successfully applied in various graph-based tasks. In a typical graph convolutional layer, node features are updated by aggregating neighborhood information. Repeatedly applying graph convolutions can cause the oversmoothing issue, i.e., node features at deep layers converge to similar values. Previous studies have suggested that oversmoothing is one of the major issues that restrict the performance of GCNs. In this article, we propose a stochastic regularization method to tackle the oversmoothing problem. In the proposed method, we stochastically scale features and gradients (SSFG) by a factor sampled from a probability distribution in the training procedure. By explicitly applying a scaling factor to break feature convergence, the oversmoothing issue is alleviated. We show that applying stochastic scaling at the gradient level is complementary to that applied at the feature level to improve the overall performance. Our method does not increase the number of trainable parameters. When used together with ReLU, our SSFG can be seen as a stochastic ReLU activation function. We experimentally validate our SSFG regularization method on three commonly used types of graph networks. Extensive experimental results on seven benchmark datasets for four graph-based tasks demonstrate that our SSFG regularization is effective in improving the overall performance of the baseline graph networks. The code is available at https://github.com/vailatuts/SSFG-regularization.
Biased Complementary-Label Learning Without True Labels

Authors:
raw_author_name: Jianfei Ruan
raw_affiliation_string: School of Computer Science and Technology, and the Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi&#x2019;an Jiaotong University, Xi&#x2019;an, Shaanxi, China
raw_author_name: Qinghua Zheng
raw_affiliation_string: School of Computer Science and Technology, and the Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi&#x2019;an Jiaotong University, Xi&#x2019;an, Shaanxi, China
raw_author_name: Rui Zhao
raw_affiliation_string: School of Computer Science and Technology, and the Ministry of Education Key Laboratory for Intelligent Networks and Network Security, Xi&#x2019;an Jiaotong University, Xi&#x2019;an, Shaanxi, China
raw_author_name: Bo Dong
raw_affiliation_string: School of Continuing Education, and the Shaanxi Province Key Laboratory of Satellite and Terrestrial Network Technology Research and Development, Xi&#x2019;an Jiaotong University, Xi&#x2019;an, Shaanxi, China

Abstract: In complementary-label learning (CLL), the complementary transition matrix, denoting the probabilities that true labels flip into complementary labels (CLs) which specify classes observations do not belong to, is crucial to building statistically consistent classifiers. Most existing works implicitly assume that the transition probabilities are identical, which is not true in practice and may lead to undesirable bias in solutions. Few recent works have extended the problem to a biased setting but limit their explorations to modeling the transition matrix by exploiting the complementary class posteriors of anchor points (i.e., instances that almost certainly belong to a specific class). However, due to the severe corruption and unevenness of biased CLs, both anchor points and complementary class posteriors are difficult to predict accurately in the absence of true labels. In this article, rather than directly predicting these two error-prone items, we instead propose a divided-T estimator as an alternative to effectively learn transition matrices from only biased CLs. Specifically, we exploit semantic clustering to mitigate the adverse effects arising from CLs. By introducing the learned semantic clusters as an intermediate class, we factorize the original transition matrix into the product of two easy-to-estimate matrices that are not reliant on the two error-prone items. Both theoretical analyses and empirical results justify the effectiveness of the divided- T estimator for estimating transition matrices under a mild assumption. Experimental results on benchmark datasets further demonstrate that the divided- T estimator outperforms state-of-the-art (SOTA) methods by a substantial margin.
Riemannian Manifold-Based Feature Space and Corresponding Image Clustering Algorithms

Authors:
raw_author_name: Xuemei Zhao
raw_affiliation_string: School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin, China
raw_author_name: Chen Li
raw_affiliation_string: School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin, China
raw_author_name: Jun Wu
raw_affiliation_string: School of Electronic Engineering and Automation, Guilin University of Electronic Technology, Guilin, China
raw_author_name: Xiaoli Li
raw_affiliation_string: Center for Geospatial Information, Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

Abstract: Image feature representation is a key factor influencing the accuracy of clustering. Traditional point-based feature spaces represent spectral features of an image independently and introduce spatial relationships of pixels in the image domain to enhance the contextual information expression ability. Mapping-based feature spaces aim to preserve the structure information, but the complex computation and the unexplainability of image features have a great impact on their applications. To this end, we propose an explicit feature space called Riemannian manifold feature space (RMFS) to present the contextual information in a unified way. First, the Gaussian probability distribution function (pdf) is introduced to characterize the features of a pixel in its neighborhood system in the image domain. Then, the feature-related pdfs are mapped to a Riemannian manifold, which constructs the proposed RMFS. In RMFS, a point can express the complex contextual information of corresponding pixel in the image domain, and pixels representing the same object are linearly distributed. This gives us a chance to convert nonlinear image segmentation problems to linear computation. To verify the superiority of the expression ability of the proposed RMFS, a linear clustering algorithm and a fuzzy linear clustering algorithm are proposed. Experimental results show that the proposed RMFS-based algorithms outperform their counterparts in the spectral feature space and the RMFS-based ones without the linear distribution characteristics. This indicates that the RMFS can better express features of an image than spectral feature space, and the expressed features can be easily used to construct linear segmentation models.
Probabilistic Neural–Symbolic Models With Inductive Posterior Constraints

Authors:
raw_author_name: Ke Su
raw_affiliation_string: Department of Computer Science and Technology, THBI Laboratory, Tsinghua-Bosch Joint ML Center, Institute for AI, and the BNRist Center, Tsinghua University, Beijing, China
raw_author_name: Hang Su
raw_affiliation_string: Department of Computer Science and Technology, THBI Laboratory, Tsinghua-Bosch Joint ML Center, Institute for AI, and the BNRist Center, Tsinghua University, Beijing, China
raw_author_name: Chongxuan Li
raw_affiliation_string: Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
raw_author_name: Jun Zhu
raw_affiliation_string: Department of Computer Science and Technology, THBI Laboratory, Tsinghua-Bosch Joint ML Center, Institute for AI, and the BNRist Center, Tsinghua University, Beijing, China
raw_author_name: Bo Zhang
raw_affiliation_string: Department of Computer Science and Technology, THBI Laboratory, Tsinghua-Bosch Joint ML Center, Institute for AI, and the BNRist Center, Tsinghua University, Beijing, China

Abstract: Neural-symbolic models provide a powerful tool to tackle complex visual reasoning tasks by combining symbolic program execution for reasoning and deep representation learning for visual recognition. A probabilistic formulation of such models with stochastic latent variables can obtain an interpretable and legible reasoning system with less supervision. However, it is still nontrivial to generate reasonable symbolic structures without the guidance of domain knowledge, since it generally involves an optimization problem with both continuous and discrete variables. Despite the challenges, the interpretability of such symbolic structures provides an interface to regularize their generation by domain knowledge. In this article, we propose to incorporate the available domain knowledge into the learning process of probabilistic neural-symbolic (PNS) models via posterior constraints that directly regularize the structure posterior. In this way, our model is able to identify a middle point where the structure generation process mainly learns from data but also selectively borrows information from domain knowledge. We further present inductive reasoning where the posterior constraints can be automatically reweighted to handle noisy annotations. The experimental results show that our method achieves state-of-the-art performance on major abstract reasoning datasets and enjoys good generalization capability and data efficiency.
Teaching Agile the Agile Way — Employing Self-Organizing Teams in a University Software Engineering Course

Authors:
raw_author_name: Christoph Matthies
raw_affiliation_string:  American Society for Engineering Education, 2016; Hasso-Plattner Institute, University of Potsdam, Germany; Institute Thomas; Institute for IT Systems Engineering
raw_author_name: Matthias Uflacker
raw_affiliation_string:  American Society for Engineering Education, 2016; Hasso Plattner Institute for IT Systems Engineering; Hasso-Plattner Institute, University of Potsdam, Germany; Institute Thomas

Abstract: Kowark is a former member of the HPI research school on "Service-oriented Systems Engineering" and now a researcher at the "Enterprise Platform and Integration Concepts" group.After completing both the Bachelor's and the Master's program at HPI he decided to continue with his Ph.D. studies at this institute, but broadened the focus from a completely technical to a more design research driven level.His main focus is the replicability of
Quality Evaluation of Triples in Knowledge Graph by Incorporating Internal With External Consistency

Authors:
raw_author_name: Taiyu Ban
raw_affiliation_string: School of Computer Science and Technology, University of Science and Technology of China, Hefei, China
raw_author_name: Xiangyu Wang
raw_affiliation_string: School of Computer Science and Technology, University of Science and Technology of China, Hefei, China
raw_author_name: Xingyu Wu
raw_affiliation_string: School of Computer Science and Technology, University of Science and Technology of China, Hefei, China
raw_author_name: Qiuju Chen
raw_affiliation_string: School of Information Science and Technology, University of Science and Technology of China, Hefei, China
raw_author_name: Huanhuan Chen
raw_affiliation_string: School of Computer Science and Technology, University of Science and Technology of China, Hefei, China

Abstract: The evaluation of knowledge quality (KQ) in multisource knowledge graphs (KGs) is an essential step for many applications, such as fragmented knowledge fusion and knowledge base construction. Many existing quality evaluation methods for multisource knowledge are based on validation from high-quality knowledge bases or statistical analysis of knowledge related to a specific fact from multiple sources, named external consistency (EC)-based methods. However, high-quality KGs are difficult to obtain, and there might exist incorrect knowledge in multisource KGs interfering with KQ evaluation. To address the issue, this article refers to the internal structure of a KG to evaluate the degree to which the contained triples conform to the overall semantic pattern of the KG, such as KG embedding and logic inference-based approaches, defined as internal consistency (IC) evaluation. The IC is integrated with the EC to identify possible incorrect triples and reduce their influences on the KQ evaluation, thus alleviating the interference of incorrect knowledge. The proposed method is verified with multiple datasets, and the results demonstrate that the proposed method could significantly reduce wrong evaluations caused by incorrect knowledge and effectively improve the quality evaluation of triples.
Overview of Learning Cybersecurity Through Game Based Systems

Authors:
raw_author_name: Te-Shun Chou
raw_affiliation_string: Department of Technology Systems College of Engineering and Technology East Carolina University

Abstract: Cybersecurity awareness and skills training are very essential and challenging.Cybersecurity in itself involves the defense of systems, networks and programs from digital attacks.These attacks are capable of gaining unauthorized access to computers and networks.Game based learning allows students have fun whilst learning by actively learning and practicing the right ways things should be done.Often, the game is started on a slow pace gradually advancing gain in skill until the student is able to successfully navigate the difficult levels.There is a constant increase in cyberattacks all over the world, an estimate of $106 Billion was recorded for cyber hacks in the United States in 2016 alone.The cybersecurity skills shortage is also posing a major concern.Hence, it has become imperative to develop a learning platform for the next generation of cybersecurity professionals to learn and be further equipped by introducing cybersecurity with the concept of gaming.Some of the games developed offer some challenges and a higher level of thinking.Learning through this system introduces an immersive, learner-centered experience with effectiveness on cybersecurity awareness training and practical skill acquisition for learners from diverse backgrounds.This paper evaluates the different game based learning system in cybersecurity, identifying the various tools in existence, the benefits and shortcomings of each system and feasible ways to improve on the existing systems.
ACERAC: Efficient Reinforcement Learning in Fine Time Discretization

Authors:
raw_author_name: Pawel Wawrzynski
raw_affiliation_string: Institute of Computer Science, Warsaw University of Technology, Warsaw, Poland
raw_author_name: Jakub Lyskawa
raw_affiliation_string: Institute of Computer Science, Warsaw University of Technology, Warsaw, Poland

Abstract: One of the main goals of reinforcement learning (RL) is to provide a way for physical machines to learn optimal behavior instead of being programmed. However, effective control of the machines usually requires fine time discretization. The most common RL methods apply independent random elements to each action, which is not suitable in that setting. It is not feasible because it causes the controlled system to jerk and does not ensure sufficient exploration since a single action is not long enough to create a significant experience that could be translated into policy improvement. In our view, these are the main obstacles that prevent the application of RL in contemporary control systems. To address these pitfalls, in this article, we introduce an RL framework and adequate analytical tools for actions that may be stochastically dependent in subsequent time instances. We also introduce an RL algorithm that approximately optimizes a policy that produces such actions. It applies experience replay (ER) to adjust the likelihood of sequences of previous actions to optimize expected n -step returns that the policy yields. The efficiency of this algorithm is verified against four other RL methods [continuous deep advantage updating (CDAU), proximal policy optimization (PPO), soft actor-critic (SAC), and actor-critic with ER (ACER)] in four simulated learning control problems (Ant, HalfCheetah, Hopper, and Walker2D) in diverse time discretization. The algorithm introduced here outperforms the competitors in most cases considered.
Neural Network Gaussian Processes by Increasing Depth

Authors:
raw_author_name: Shao-Qun Zhang
raw_author_name: Feng-Lei Fan
raw_affiliation_string: Rensselaer  Polytechnic Institute

Abstract: Recent years have witnessed an increasing interest in the correspondence between infinitely wide networks and Gaussian processes. Despite the effectiveness and elegance of the current neural network Gaussian process theory, to the best of our knowledge, all the neural network Gaussian processes (NNGPs) are essentially induced by increasing width. However, in the era of deep learning, what concerns us more regarding a neural network is its depth as well as how depth impacts the behaviors of a network. Inspired by a width-depth symmetry consideration, we use a shortcut network to show that increasing the depth of a neural network can also give rise to a Gaussian process, which is a valuable addition to the existing theory and contributes to revealing the true picture of deep learning. Beyond the proposed Gaussian process by depth, we theoretically characterize its uniform tightness property and the smallest eigenvalue of the Gaussian process kernel. These characterizations can not only enhance our understanding of the proposed depth-induced Gaussian process but also pave the way for future applications. Lastly, we examine the performance of the proposed Gaussian process by regression experiments on two benchmark datasets.
Visual Exploration of Machine Learning Model Behavior With Hierarchical Surrogate Rule Sets

Authors:
raw_author_name: Yuan, Jun
raw_affiliation_string: New York University, New York, NY, USA
raw_author_name: Barr, Brian
raw_affiliation_string: Capital One, McLean, VA, USA
raw_author_name: Overton, Kyle
raw_affiliation_string: Capital One, McLean, VA, USA
raw_author_name: Bertini, Enrico
raw_affiliation_string: Northeastern University, Boston, MA, USA

Abstract: One of the potential solutions for model interpretation is to train a surrogate model: a more transparent model that approximates the behavior of the model to be explained. Typically, classification rules or decision trees are used due to their logic-based expressions. However, decision trees can grow too deep, and rule sets can become too large to approximate a complex model. Unlike paths on a decision tree that must share ancestor nodes (conditions), rules are more flexible. However, the unstructured visual representation of rules makes it hard to make inferences across rules. In this paper, we focus on tabular data and present novel algorithmic and interactive solutions to address these issues. First, we present Hierarchical Surrogate Rules (HSR), an algorithm that generates hierarchical rules based on user-defined parameters. We also contribute SuRE, a visual analytics (VA) system that integrates HSR and an interactive surrogate rule visualization, the Feature-Aligned Tree, which depicts rules as trees while aligning features for easier comparison. We evaluate the algorithm in terms of parameter sensitivity, time performance, and comparison with surrogate decision trees and find that it scales reasonably well and overcomes the shortcomings of surrogate decision trees. We evaluate the visualization and the system through a usability study and an observational study with domain experts. Our investigation shows that the participants can use feature-aligned trees to perform non-trivial tasks with very high accuracy. We also discuss many interesting findings, including a rule analysis task characterization, that can be used for visualization design and future research.
Adaptive Semantic-Enhanced Transformer for Image Captioning

Authors:
raw_author_name: Jing Zhang
raw_affiliation_string: Department of Computer Science and Technology, East China University of Science and Technology, Shanghai, China
raw_author_name: Han Sun
raw_affiliation_string: Department of Computer Science and Technology, East China University of Science and Technology, Shanghai, China
raw_author_name: Zhe Wang
raw_affiliation_string: Department of Computer Science and Technology, East China University of Science and Technology, Shanghai, China

Abstract: In the research on image captioning, rich semantic information is very important for generating critical caption words as guiding information. However, semantic information from offline object detectors involves many semantic objects that do not appear in the caption, thereby bringing noise into the decoding process. To produce more accurate semantic guiding information and further optimize the decoding process, we propose an end-to-end adaptive semantic-enhanced transformer (AS-Transformer) model for image captioning. For semantic enhancement information extraction, we propose a constrained weaklysupervised learning (CWSL) module, which reconstructs the semantic object's probability distribution detected by the multiple instances learning (MIL) through a joint loss function. These strengthened semantic objects from the reconstructed probability distribution can better depict the semantic meaning of images. Also, for semantic enhancement decoding, we propose an adaptive gated mechanism (AGM) module to adjust the attention between visual and semantic information adaptively for the more accurate generation of caption words. Through the joint control of the CWSL module and AGM module, our proposed model constructs a complete adaptive enhancement mechanism from encoding to decoding and obtains visual context that is more suitable for captions. Experiments on the public Microsoft Common Objects in COntext (MSCOCO) and Flickr30K datasets illustrate that our proposed AS-Transformer can adaptively obtain effective semantic information and adjust the attention weights between semantic and visual information automatically, which achieves more accurate captions compared with semantic enhancement methods and outperforms state-of-the-art methods.