Reviewerly/exHarmony huggingface dataset:
User-Oriented Robust Reinforcement Learning

Recently, improving the robustness of policies across different environments attracts increasing attention in the reinforcement learning (RL) community. Existing robust RL methods mostly aim to achieve the max-min robustness by optimizing the policy’s performance in the worst-case environment. However, in practice, a user that uses an RL policy may have different preferences over its performance across environments. Clearly, the aforementioned max-min robustness is oftentimes too conservative to satisfy user preference. Therefore, in this paper, we integrate user preference into policy learning in robust RL, and propose a novel User-Oriented Robust RL (UOR-RL) framework. Specifically, we define a new User-Oriented Robustness (UOR) metric for RL, which allocates different weights to the environments according to user preference and generalizes the max-min robustness metric. To optimize the UOR metric, we develop two different UOR-RL training algorithms for the scenarios with or without a priori known environment distribution, respectively. Theoretically, we prove that our UOR-RL training algorithms converge to near-optimal policies even with inaccurate or completely no knowledge about the environment distribution. Furthermore, we carry out extensive experimental evaluations in 6 MuJoCo tasks. The experimental results demonstrate that UOR-RL is comparable to the state-of-the-art baselines under the average-case and worst-case performance metrics, and more importantly establishes new state-of-the-art performance under the UOR metric.
Failure-Resistant Intelligent Interaction for Reliable Human-AI Collaboration

My thesis is focusing on how we can overcome the gap people have against machine learning techniques that require a well-defined application scheme and can produce wrong results. I am planning to discuss the principle of the interaction design that fills such a gap based on my past projects that have explored better interactions for applying machine learning in various fields, such as malware analysis, executive coaching, photo editing, and so on. To this aim, my thesis also shed a light on the limitations of machine learning techniques, like adversarial examples, to highlight the importance of "failure-resistant intelligent interaction."
Fuzzy C-means: Differences on Clustering Behavior between High Dimensional and Functional Data (Student Abstract)

Fuzzy c-means (FCM) is a generalization of the classical k-means clustering algorithm to the case where an observation can belong to several clusters at the same time. The algorithm was previously observed to have initialization problems when the number of desired clusters or the number of dimensions of the data are high. We have tested FCM against clustering problems with functional data, generated from stationary Gaussian processes, and thus in principle infinite-dimensional. We observed that when the data is more functional in nature, which can be obtained by tuning the length-scale parameter of the Gaussian process, the aforementioned problems do not appear. This not only indicates that FCM is suitable as a clustering method for functional data, but also illustrates how functional data differs from traditional multivariate data. In addition this seems to suggest a qualitative way to measure the latent dimensionality of the functional distribution itself.
ConceptX: A Framework for Latent Concept Analysis

The opacity of deep neural networks remains a challenge in deploying solutions where explanation is as important as precision. We present ConceptX, a human-in-the-loop framework for interpreting and annotating latent representational space in pre-trained Language Models (pLMs). We use an unsupervised method to discover concepts learned in these models and enable a graphical interface for humans to generate explanations for the concepts. To facilitate the process, we provide auto-annotations of the concepts (based on traditional linguistic ontologies). Such annotations enable development of a linguistic resource that directly represents latent concepts learned within deep NLP models. These include not just traditional linguistic concepts, but also task-specific or sensitive concepts (words grouped based on gender or religious connotation) that helps the annotators to mark bias in the model. The framework consists of two parts (i) concept discovery and (ii) annotation platform.
CMVAE: Causal Meta VAE for Unsupervised Meta-Learning

Unsupervised meta-learning aims to learn the meta knowledge from unlabeled data and rapidly adapt to novel tasks. However, existing approaches may be misled by the context-bias (e.g. background) from the training data. In this paper, we abstract the unsupervised meta-learning problem into a Structural Causal Model (SCM) and point out that such bias arises due to hidden confounders. To eliminate the confounders, we define the priors are conditionally independent, learn the relationships between priors and intervene on them with casual factorization. Furthermore, we propose Causal Meta VAE (CMVAE) that encodes the priors into latent codes in the causal space and learns their relationships simultaneously to achieve the downstream few-shot image classification task. Results on toy datasets and three benchmark datasets demonstrate that our method can remove the context-bias and it outperforms other state-of-the-art unsupervised meta-learning algorithms because of bias-removal. Code is available at https://github.com/GuodongQi/CMVAE.
Scalable Bayesian Meta-Learning through Generalized Implicit Gradients

Meta-learning owns unique effectiveness and swiftness in tackling emerging tasks with limited data. Its broad applicability is revealed by viewing it as a bi-level optimization problem. The resultant algorithmic viewpoint however, faces scalability issues when the inner-level optimization relies on gradient-based iterations. Implicit differentiation has been considered to alleviate this challenge, but it is restricted to an isotropic Gaussian prior, and only favors deterministic meta-learning approaches. This work markedly mitigates the scalability bottleneck by cross-fertilizing the benefits of implicit differentiation to probabilistic Bayesian meta-learning. The novel implicit Bayesian meta-learning (iBaML) method not only broadens the scope of learnable priors, but also quantifies the associated uncertainty. Furthermore, the ultimate complexity is well controlled regardless of the inner-level optimization trajectory. Analytical error bounds are established to demonstrate the precision and efficiency of the generalized implicit gradient over the explicit one. Extensive numerical tests are also carried out to empirically validate the performance of the proposed method.
Redactor: A Data-Centric and Individualized Defense against Inference Attacks

Information leakage is becoming a critical problem as various information becomes publicly available by mistake, and machine learning models train on that data to provide services. As a result, one's private information could easily be memorized by such trained models. Unfortunately, deleting information is out of the question as the data is already exposed to the Web or third-party platforms. Moreover, we cannot necessarily control the labeling process and the model trainings by other parties either. In this setting, we study the problem of targeted disinformation generation where the goal is to dilute the data and thus make a model safer and more robust against inference attacks on a specific target (e.g., a person's profile) by only inserting new data. Our method finds the closest points to the target in the input space that will be labeled as a different class. Since we cannot control the labeling process, we instead conservatively estimate the labels probabilistically by combining decision boundaries of multiple classifiers using data programming techniques. Our experiments show that a probabilistic decision boundary can be a good proxy for labelers, and that our approach is effective in defending against inference attacks and can scale to large data.
Feature-Space Bayesian Adversarial Learning Improved Malware Detector Robustness

We present a new algorithm to train a robust malware detector. Malware is a prolific problem and malware detectors are a front-line defense. Modern detectors rely on machine learning algorithms. Now, the adversarial objective is to devise alterations to the malware code to decrease the chance of being detected whilst preserving the functionality and realism of the malware. Adversarial learning is effective in improving robustness but generating functional and realistic adversarial malware samples is non-trivial. Because: i) in contrast to tasks capable of using gradient-based feedback, adversarial learning in a domain without a differentiable mapping function from the problem space (malware code inputs) to the feature space is hard; and ii) it is difficult to ensure the adversarial malware is realistic and functional. This presents a challenge for developing scalable adversarial machine learning algorithms for large datasets at a production or commercial scale to realize robust malware detectors. We propose an alternative; perform adversarial learning in the feature space in contrast to the problem space. We prove the projection of perturbed, yet valid malware, in the problem space into feature space will always be a subset of adversarials generated in the feature space. Hence, by generating a robust network against feature-space adversarial examples, we inherently achieve robustness against problem-space adversarial examples. We formulate a Bayesian adversarial learning objective that captures the distribution of models for improved robustness. To explain the robustness of the Bayesian adversarial learning algorithm, we prove that our learning method bounds the difference between the adversarial risk and empirical risk and improves robustness. We show that Bayesian neural networks (BNNs) achieve state-of-the-art results; especially in the False Positive Rate (FPR) regime. Adversarially trained BNNs achieve state-of-the-art robustness. Notably, adversarially trained BNNs are robust against stronger attacks with larger attack budgets by a margin of up to 15% on a recent production-scale malware dataset of more than 20 million samples. Importantly, our efforts create a benchmark for future defenses in the malware domain.
Interventional SHAP Values and Interaction Values for Piecewise Linear Regression Trees

In recent years, game-theoretic Shapley values have gained increasing attention with respect to local model explanation by feature attributions. While the approach using Shapley values is model-independent, their (exact) computation is usually intractable, so efficient model-specific algorithms have been devised including approaches for decision trees or their ensembles in general. Our work goes further in this direction by extending the interventional TreeSHAP algorithm to piecewise linear regression trees, which gained more attention in the past few years. To this end, we introduce a decomposition of the contribution function based on decision paths, which allows a more comprehensible formulation of SHAP algorithms for tree-based models. Our algorithm can also be readily applied to computing SHAP interaction values of these models. In particular, as the main contribution of this paper, we provide a more efficient approach of interventional SHAP for tree-based models by precomputing statistics of the background data based on the tree structure.
Double Policy Network for Aspect Sentiment Triplet Extraction (Student Abstract)

Aspect Sentiment Triplet Extraction (ASTE) is the task to extract aspects, opinions and associated sentiments from sentences. Previous studies do not adequately consider the complicated interactions between aspect and opinion terms in both extraction logic and strategy. We present a novel Double Policy Network with Multi-Tag based Reward model (DPN-MTR), which adopts two networks ATE, TSOTE and a Trigger Mechanism to execute ASTE task following a more logical framework. A Multi-Tag based reward is also proposed to solve the limitations of existing studies for identifying aspect/opinion terms with multiple tokens (one term may consist of two or more tokens) to a certain extent. Extensive experiments are conducted on four widely-used benchmark datasets, and demonstrate the effectiveness of our model in generally improving the performance on ASTE significantly.
Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks

Gate functions in recurrent models, such as an LSTM and GRU, play a central role in learning various time scales in modeling time series data by using a bounded activation function. However, it is difficult to train gates to capture extremely long time scales due to gradient vanishing of the bounded function for large inputs, which is known as the saturation problem. We closely analyze the relation between saturation of the gate function and efficiency of the training. We prove that the gradient vanishing of the gate function can be mitigated by accelerating the convergence of the saturating function, i.e., making the output of the function converge to 0 or 1 faster. Based on the analysis results, we propose a gate function called fast gate that has a doubly exponential convergence rate with respect to inputs by simple function composition. We empirically show that our method outperforms previous methods in accuracy and computational efficiency on benchmark tasks involving extremely long time scales.
Mitigating Negative Transfer in Multi-Task Learning with Exponential Moving Average Loss Weighting Strategies (Student Abstract)

Multi-Task Learning (MTL) is a growing subject of interest in deep learning, due to its ability to train models more efficiently on multiple tasks compared to using a group of conventional single-task models. However, MTL can be impractical as certain tasks can dominate training and hurt performance in others, thus making some tasks perform better in a single-task model compared to a multi-task one. Such problems are broadly classified as negative transfer, and many prior approaches in the literature have been made to mitigate these issues. One such current approach to alleviate negative transfer is to weight each of the losses so that they are on the same scale. Whereas current loss balancing approaches rely on either optimization or complex numerical analysis, none directly scale the losses based on their observed magnitudes. We propose multiple techniques for loss balancing based on scaling by the exponential moving average and benchmark them against current best-performing methods on three established datasets. On these datasets, they achieve comparable, if not higher, performance compared to current best-performing methods.
Understanding Representation Learnability of Nonlinear Self-Supervised Learning

Self-supervised learning (SSL) has empirically shown its data representation learnability in many downstream tasks. There are only a few theoretical works on data representation learnability, and many of those focus on final data representation, treating the nonlinear neural network as a ``black box". However, the accurate learning results of neural networks are crucial for describing the data distribution features learned by SSL models. Our paper is the first to analyze the learning results of the nonlinear SSL model accurately. We consider a toy data distribution that contains two features: the label-related feature and the hidden feature. Unlike previous linear setting work that depends on closed-form solutions, we use the gradient descent algorithm to train a 1-layer nonlinear SSL model with a certain initialization region and prove that the model converges to a local minimum. Furthermore, different from the complex iterative analysis, we propose a new analysis process which uses the exact version of Inverse Function Theorem to accurately describe the features learned by the local minimum. With this local minimum, we prove that the nonlinear SSL model can capture the label-related feature and hidden feature at the same time. In contrast, the nonlinear supervised learning (SL) model can only learn the label-related feature. We also present the learning processes and results of the nonlinear SSL and SL model via simulation experiments.
DIAGNOSTICS OF THE FORMATION OF THE COMPETENCE OF PEDAGOGICAL PARTNERSHIP AMONG PRIMARY SCHOOL TEACHERS

The article reveals the state of formation of partnership pedagogy competence in accordance with the “New Ukrainian School” Concept and pedagogical standards of primary school teachers.The purpose of the article is to highlight the state of formation of partnership pedagogy competence among primary school teachers. In determining the competence of partnership pedagogy, we relied on cognitive, motivational-value, and practical-activity criteria. The role of a modern primary school teacher in the context of building his partnership with students and parents is outlined. The expediency of partnership with parents as important subjects of the educational process is emphasized.The pedagogy of partnership is substantiated as a factor in the formation of effective interaction of participants in the educational process based on the principles of pedagogy of partnership. The results of the research on the level of formation of partnership pedagogy competence of primary school teachers in interaction with participants of the educational process based on partnership pedagogy are presented.The leading essential feature of partnership pedagogy is proven, which is cooperation between the teacher, students and parents as equal participants in the educational process in a modern school.The conclusion is substantiated that an important task of the New Ukrainian School is a new format of partnership interaction between the teacher and parents, which allows defining a common vision of the child's development trajectory.The obtained results will help to adjust the process of improving the competence formation of partnership pedagogy among primary school teachers.The practical significance of the article is that its materials can be used in the organization of the educational process of general education institutions and primary school teacher training.&#x0D; Keywords: New Ukrainian school; competence; partnership pedagogy; criteria; indicators; interaction; participants in the educational process.
Politeness strategies used by Papuan students in virtual communication practices at university

This study aims at investigating and revealing language politeness strategies, as well as the factors that influence politeness strategies used by Papuan students during virtual in the context of virtual academic consultations as a practice of virtual communications to the lecturers. This study uses descriptive qualitative methods and purposive sampling to choose two lecturers and fifteen university students from the Papua as participants. Data was also gathered through online observation, offline interviews, and documentation. Online observations were made by evaluating screenshots of Google Classroom comments and recording of written conversations on WhatsApp and SMS sent by ten Papuan students. At the same time, offline interviews were carried out 10 students and 2 instructors. The data is then analysed using discourse analysis, which consists of multiple steps, including data recording, data transcription, data selection, and data interpretation. In line with the findings of this study, Papuan students use four types of politeness strategies when consulting with lecturers via virtual communication. They are positive, negative, bald on record, and off the record politeness strategies. In online communication, the negative politeness strategy is the most prevalent among Papuan students. The diversity of politeness strategies that are replicated, even if they appear unpleasant at times, is not an aspect of purposeful or disrespectful behaviour toward the interlocutor. But let us get back to the level of language skills, which still require more attention and are impacted by a variety of internal factors such as personality and age, and external factors such as culture, facilities obtained. The findings indicate that even when language capabilities are restricted in virtual communication, Papuan students receive enough education. This situation will improve the civility of language that students in Papua use while conversing digitally. As a result, higher education institutions in Papua should pay more attention to standard communication patterns when academic consulting with lecturers as speech partners in the context of higher education in Papua in order to develop comprehensive and effective communication practices.
THEORETICAL ISSUES OF THE TRAINING OF FUTURE PRIMARY SCHOOL TEACHERS FOR THE USE OF INNOVATIVE TECHNOLOGIES IN THEIR PROFESSIONAL ACTIVITIES

The article deals with the raising of requirements for the professional training of future primary school teachers. This predetermines qualitatively new theoretical and methodical approaches to the training of students, which will allow them not only to master the basics of science and professional skills and abilities, but also new pedagogical achievements, innovative technologies etc. while studying in higher pedagogical institutions Successful mastering of innovative pedagogical technologies helps bachelors, specialists and masters meaningfully start the process of independent professional activity. The long-term experience of scientists in the training of future primary school teachers gives reason to assert that the solution to the problem of person-oriented education and its productivity is connected with the use of innovative, in particular, interactive technologies in the educational process. Scientific research show that interaction in many ways solves the problem of strengthening and preserving the mental and physical health of students, contributes to increasing the motivation towards educational activities. Interactive technologies in all their diversity are focused on realization of cognitive interests and needs of an individual, therefore special attention is given to organization of the process of effective communication, in which the participants of the interaction process are more mobile, more open and active. Modern higher education should address the principle of multilateral communication, and such its organization promotes the use of appropriate teaching methods. This contributes to the fact that students acquire sufficient knowledge of didactics, professional methods of primary education etc. for the implementation of innovative education of primary schoolchildren. The future scientific search should be the introduction of interactive technologies in the educational process of training future primary school teachers.
Face Recognition System for Real Life Applications using Augmentation Techniques

A facial recognition system may recognise a person by comparing his face to the ones in a database of faces taken from a digital image or video frame. It is primarily used to verify the identity of users by analysing facial features from an image. According to several studies, facial recognition technology has made enormous strides with the growth of science and technology, but there is still opportunity for its improvement in practical use. Some issues like rotation, occlusion, and meta learning approach can be handled for improved model accuracy by using 3D technologies like image augmentation to supplement 2D images. Today's facial recognition technology can be used for more than only identifying people; it can also be used for security purposes in residential, business, and commercial applications. Due to its convenience, face recognition technology is widely employed in the financial sector as well. This research proposes face recognition for real life applications using Siamese network and Deep Neural Network, Facenet.