Reviewerly/exHarmony huggingface dataset:
Optimization and Innovation of English Major Teaching Model Based on Micro-class

In order to improve the teaching effect of English majors, a research on the optimization and innovation of the teaching mode of English majors based on micro-classes is proposed.Analyzed the basic attributes of micro-classes in the way of expression and application, combined with the two problems of teachers' inadequate cognition of micro-classes and limited space for micro-class creation, and put forward the basic principles of insisting on teaching without class.Pay attention to teaching students in accordance with their aptitude, strengthen heuristic teaching, and pay attention to the optimization and innovation path with the combination of learning and thinking as the core.
A study of sentiment differences in different event-related texts based on natural language processing

Which kinds of events can bring more happiness to us? Which types of events are more likely to make people sad? Are those events related to life, relationships, work, or study? Some clues can be found by analyzing many texts recorded by myriad number of users on social media. This study used natural language processing (NLP) techniques to mine important information from 80,747 text-based Weibo blog posts and determined their textual sentiment values. SPSS statistical software was also used to analyze the variance in sentiment values of life, relationships, work, and study related texts. The results showed that work-related texts had the highest proportion of negative sentiment values and a significantly lower mean sentiment value, and that study-related texts had a significantly higher mean sentiment value.
A Study of Middle School English Grammar Teaching in the CLT Framework

As an essential part of language, grammar plays an important role in the process of language learning.Grammar teaching is a heatedly discussed topic in foreign language teaching circle.Since the application of CLT in China, grammar teaching has caused more concern than before.This thesis illustrates the necessity of grammar teaching in the theoretical and practical perspectives.In theory, the goal of CLT is to cultivate students' communicative competence and grammatical competence is part of it, so CLT does not exclude grammar teaching but include it.In practice, Chinese realistic conditions determine that grammar should be taught in the CLT framework.Then this thesis proposes some strategies to improve grammar teaching, which tends to give some enlightenment to English teachers.
Preface: The 1st International Conference of Mathematics Education, Learning and Application 2021

First Page
The development of a mobile phone application based on RME model for probability of union of two events

This research is the third of three years fully funded and supported by the National Research and Innovation Agency for the 2019-2021 fiscal year. This study resulted in a mobile learning application design for the probability of the union of two events based on the RME model on mobile devices. The reference for developing this application is an RME-based high school mathematics learning guide book, especially probability material. Application development goes through nine stages, namely: (1) storyboard design; (2) graphic design; (3) animation design; (4) voiceover design; (5) animation video design; (6) consultation with mathematics education experts and teachers; (7) first revision based on suggestions and input from mathematicians and teachers; (7) trial use of the application on several students; (8) second revision based on student input. The average scores of two experts and two mathematics teachers were 93 for display, 89 for text, 94 for device engineering, and 87 for sound. Both teachers and their students love to use this app for learning. The final product of the research is an application that is available for free on the Google PlayStore and can be downloaded at: https://play.google.com/store/apps/details?id=com.uhamka.application_rme.
Real Time Crowd Analysis and Anomaly Detection

Abstract Crowd analysis is gaining a lot of attention in both the technological and social aspects as the population expands. As certain crowdactivity cannot be assessed easily, These techniques differ dependingon the crowd. Strikes, protests, and other crowded events are commonin overpopulated cities. Parades and other public gatherings providea variety of security challenges. Security forces are frequently enlistedto keep an eye on these issues and ensuring the safety of those whoattend meetings. In these circumstances, ensuring the safety of thosewho attend meetings are challenging . Cutting-edge crowd control methods are necessary for an intelligent surveillance system. The goal of theproposed method is to find crowd analysis subfields, such as anomalydetection, that have not yet been well explored or that seem to beaddressed infrequently. Public data sets for indoor and outdoor useare used to evaluate the proposed system’s bespoke architecture. The Experimental findings demonstrate that the custom architecture performs better than the exixting Architectures.Our aim is to providerecent state-of-the-art algorithms to provide a novel strategy that canhelp with improved crowd surveillance and crowd activity analysis.
Robust Transferable Feature Extractors: Learning to Defend Pre-Trained Networks Against White Box Adversaries

The widespread adoption of deep neural networks in computer vision applications has brought forth a significant interest in adversarial robustness. Existing research has shown that maliciously perturbed inputs specifically tailored for a given model (i.e., adversarial examples) can be successfully transferred to another independently trained model to induce prediction errors. Moreover, this property of adversarial examples has been attributed to features derived from predictive patterns in the data distribution. Thus, we are motivated to investigate the following question: Can adversarial defenses, like adversarial examples, be successfully transferred to other independently trained models? To this end, we propose a deep learning-based pre-processing mechanism, which we refer to as a robust transferable feature extractor (RTFE). After examining theoretical motivation and implications, we experimentally show that our method can provide adversarial robustness to multiple independently pre-trained classifiers that are otherwise ineffective against an adaptive white box adversary. Furthermore, we show that RTFEs can even provide one-shot adversarial robustness to models independently trained on different datasets.
Joint Debiased Representation and Image Clustering Learning with Self-Supervision

Contrastive learning is among the most successful methods for visual representation learning, and its performance can be further improved by jointly performing clustering on the learned representations. However, existing methods for joint clustering and contrastive learning do not perform well on long-tailed data distributions, as majority classes overwhelm and distort the loss of minority classes, thus preventing meaningful representations to be learned. Motivated by this, we develop a novel joint clustering and contrastive learning framework by adapting the debiased contrastive loss to avoid under-clustering minority classes of imbalanced datasets. We show that our proposed modified debiased contrastive loss and divergence clustering loss improves the performance across multiple datasets and learning tasks. The source code is available at https://anonymous.4open.science/r/SSL-debiased-clustering
M^4I: Multi-modal Models Membership Inference

With the development of machine learning techniques, the attention of research has been moved from single-modal learning to multi-modal learning, as real-world data exist in the form of different modalities. However, multi-modal models often carry more information than single-modal models and they are usually applied in sensitive scenarios, such as medical report generation or disease identification. Compared with the existing membership inference against machine learning classifiers, we focus on the problem that the input and output of the multi-modal models are in different modalities, such as image captioning. This work studies the privacy leakage of multi-modal models through the lens of membership inference attack, a process of determining whether a data record involves in the model training process or not. To achieve this, we propose Multi-modal Models Membership Inference (M^4I) with two attack methods to infer the membership status, named metric-based (MB) M^4I and feature-based (FB) M^4I, respectively. More specifically, MB M^4I adopts similarity metrics while attacking to infer target data membership. FB M^4I uses a pre-trained shadow multi-modal feature extractor to achieve the purpose of data inference attack by comparing the similarities from extracted input and output features. Extensive experimental results show that both attack methods can achieve strong performances. Respectively, 72.5% and 94.83% of attack success rates on average can be obtained under unrestricted scenarios. Moreover, we evaluate multiple defense mechanisms against our attacks. The source code of M^4I attacks is publicly available at https://github.com/MultimodalMI/Multimodal-membership-inference.git.
Stochastic Tree Ensembles for Estimating Heterogeneous Effects

Determining subgroups that respond especially well (or poorly) to specific interventions (medical or policy) requires new supervised learning methods tailored specifically for causal inference. Bayesian Causal Forest (BCF) is a recent method that has been documented to perform well on data generating processes with strong confounding of the sort that is plausible in many applications. This paper develops a novel algorithm for fitting the BCF model, which is more efficient than the previously available Gibbs sampler. The new algorithm can be used to initialize independent chains of the existing Gibbs sampler leading to better posterior exploration and coverage of the associated interval estimates in simulation studies. The new algorithm is compared to related approaches via simulation studies as well as an empirical analysis.
GAGA: Deciphering Age-path of Generalized Self-paced Regularizer

Nowadays self-paced learning (SPL) is an important machine learning paradigm that mimics the cognitive process of humans and animals. The SPL regime involves a self-paced regularizer and a gradually increasing age parameter, which plays a key role in SPL but where to optimally terminate this process is still non-trivial to determine. A natural idea is to compute the solution path w.r.t. age parameter (i.e., age-path). However, current age-path algorithms are either limited to the simplest regularizer, or lack solid theoretical understanding as well as computational efficiency. To address this challenge, we propose a novel \underline{G}eneralized \underline{Ag}e-path \underline{A}lgorithm (GAGA) for SPL with various self-paced regularizers based on ordinary differential equations (ODEs) and sets control, which can learn the entire solution spectrum w.r.t. a range of age parameters. To the best of our knowledge, GAGA is the first exact path-following algorithm tackling the age-path for general self-paced regularizer. Finally the algorithmic steps of classic SVM and Lasso are described in detail. We demonstrate the performance of GAGA on real-world datasets, and find considerable speedup between our algorithm and competing baselines.
Efficient learning of nonlinear prediction models with time-series privileged information

In domains where sample sizes are limited, efficient learning algorithms are critical. Learning using privileged information (LuPI) offers increased sample efficiency by allowing prediction models access to auxiliary information at training time which is unavailable when the models are used. In recent work, it was shown that for prediction in linear-Gaussian dynamical systems, a LuPI learner with access to intermediate time series data is never worse and often better in expectation than any unbiased classical learner. We provide new insights into this analysis and generalize it to nonlinear prediction tasks in latent dynamical systems, extending theoretical guarantees to the case where the map connecting latent variables and observations is known up to a linear transform. In addition, we propose algorithms based on random features and representation learning for the case when this map is unknown. A suite of empirical results confirm theoretical findings and show the potential of using privileged time-series information in nonlinear prediction.
Measuring Tail Risks

Value at risk (VaR) and expected shortfall (ES) are common high quantile-based risk measures adopted in financial regulations and risk management. In this paper, we propose a tail risk measure based on the most probable maximum size of risk events (MPMR) that can occur over a length of time. MPMR underscores the dependence of the tail risk on the risk management time frame. Unlike VaR and ES, MPMR does not require specifying a confidence level. We derive the risk measure analytically for several well-known distributions. In particular, for the case where the size of the risk event follows a power law or Pareto distribution, we show that MPMR also scales with the number of observations $n$ (or equivalently the length of the time interval) by a power law, $\text{MPMR}(n) \propto n^η$, where $η$ is the scaling exponent. The scale invariance allows for reasonable estimations of long-term risks based on the extrapolation of more reliable estimations of short-term risks. The scaling relationship also gives rise to a robust and low-bias estimator of the tail index (TI) $ξ$ of the size distribution, $ξ= 1/η$. We demonstrate the use of this risk measure for describing the tail risks in financial markets as well as the risks associated with natural hazards (earthquakes, tsunamis, and excessive rainfall).
Enhancements in cross-temporal forecast reconciliation, with an application to solar irradiance forecasts

In recent works by Yang et al. (2017a,b), and Yagli et al. (2019), geographical, temporal, and sequential deterministic reconciliation of hierarchical photovoltaic (PV) power generation have been considered for a simulated PV dataset in California. In the first two cases, the reconciliations are carried out in spatial and temporal domains separately. To further improve forecasting accuracy, in the third case these two reconciliation approaches are sequentially applied. During the replication of the forecasting experiment, some issues emerged about non-negativity and coherence (in space and/or in time) of the sequentially reconciled forecasts. Furthermore, while the accuracy improvement of the considered approaches over the benchmark persistence forecasts is clearly visible at any data granularity, we argue that an even better performance may be obtained by a thorough exploitation of cross-temporal hierarchies. In this paper the cross-temporal point forecast reconciliation approach is applied to generate non-negative, fully coherent (both in space and time) forecasts. In particular, some relationships between two-step, iterative and simultaneous cross-temporal reconciliation procedures are for the first time established, non-negativity issues of the final reconciled forecasts are correctly dealt with in a simple way, and the most recent cross-temporal reconciliation approaches are adopted. The normalised Root Mean Square Error is used to measure forecasting accuracy, and a statistical multiple comparison procedure is performed to rank the approaches. Besides assuring full coherence, and non-negativity of the reconciled forecasts, the results show that for the considered dataset, cross-temporal forecast reconciliation significantly improves on the sequential procedures proposed by Yagli et al. (2019), at any cross-sectional level of the hierarchy and for any temporal granularity.
On the detrimental effect of invariances in the likelihood for variational inference

Variational Bayesian posterior inference often requires simplifying approximations such as mean-field parametrisation to ensure tractability. However, prior work has associated the variational mean-field approximation for Bayesian neural networks with underfitting in the case of small datasets or large model sizes. In this work, we show that invariances in the likelihood function of over-parametrised models contribute to this phenomenon because these invariances complicate the structure of the posterior by introducing discrete and/or continuous modes which cannot be well approximated by Gaussian mean-field distributions. In particular, we show that the mean-field approximation has an additional gap in the evidence lower bound compared to a purpose-built posterior that takes into account the known invariances. Importantly, this invariance gap is not constant; it vanishes as the approximation reverts to the prior. We proceed by first considering translation invariances in a linear model with a single data point in detail. We show that, while the true posterior can be constructed from a mean-field parametrisation, this is achieved only if the objective function takes into account the invariance gap. Then, we transfer our analysis of the linear model to neural networks. Our analysis provides a framework for future work to explore solutions to the invariance problem.
Efficient first-order predictor-corrector multiple objective optimization for fair misinformation detection

Multiple-objective optimization (MOO) aims to simultaneously optimize multiple conflicting objectives and has found important applications in machine learning, such as minimizing classification loss and discrepancy in treating different populations for fairness. At optimality, further optimizing one objective will necessarily harm at least another objective, and decision-makers need to comprehensively explore multiple optima (called Pareto front) to pinpoint one final solution. We address the efficiency of finding the Pareto front. First, finding the front from scratch using stochastic multi-gradient descent (SMGD) is expensive with large neural networks and datasets. We propose to explore the Pareto front as a manifold from a few initial optima, based on a predictor-corrector method. Second, for each exploration step, the predictor solves a large-scale linear system that scales quadratically in the number of model parameters and requires one backpropagation to evaluate a second-order Hessian-vector product per iteration of the solver. We propose a Gauss-Newton approximation that only scales linearly, and that requires only first-order inner-product per iteration. This also allows for a choice between the MINRES and conjugate gradient methods when approximately solving the linear system. The innovations make predictor-corrector possible for large networks. Experiments on multi-objective (fairness and accuracy) misinformation detection tasks show that 1) the predictor-corrector method can find Pareto fronts better than or similar to SMGD with less time; and 2) the proposed first-order method does not harm the quality of the Pareto front identified by the second-order method, while further reduce running time.
Generalization Properties of NAS under Activation and Skip Connection Search

Neural Architecture Search (NAS) has fostered the automatic discovery of state-of-the-art neural architectures. Despite the progress achieved with NAS, so far there is little attention to theoretical guarantees on NAS. In this work, we study the generalization properties of NAS under a unifying framework enabling (deep) layer skip connection search and activation function search. To this end, we derive the lower (and upper) bounds of the minimum eigenvalue of the Neural Tangent Kernel (NTK) under the (in)finite-width regime using a certain search space including mixed activation functions, fully connected, and residual neural networks. We use the minimum eigenvalue to establish generalization error bounds of NAS in the stochastic gradient descent training. Importantly, we theoretically and experimentally show how the derived results can guide NAS to select the top-performing architectures, even in the case without training, leading to a train-free algorithm based on our theory. Accordingly, our numerical validation shed light on the design of computationally efficient methods for NAS. Our analysis is non-trivial due to the coupling of various architectures and activation functions under the unifying framework and has its own interest in providing the lower bound of the minimum eigenvalue of NTK in deep learning theory.
A First Application of Collaborative Learning In Particle Physics

Over the last ten years, the popularity of Machine Learning (ML) has grown exponentially in all scientific fields, including particle physics. The industry has also developed new powerful tools that, imported into academia, could revolutionise research. One recent industry development that has not yet come to the attention of the particle physics community is Collaborative Learning (CL), a framework that allows training the same ML model with different datasets. This work explores the potential of CL, testing the library Colearn with neutrino physics simulation. Colearn, developed by the British Cambridge-based firm Fetch.AI, enables decentralised machine learning tasks. Being a blockchain-mediated CL system, it allows multiple stakeholders to build a shared ML model without needing to rely on a central authority. A generic Liquid Argon Time-Projection Chamber (LArTPC) has been simulated and images produced by fictitious neutrino interactions have been used to produce several datasets. These datasets, called learners, participated successfully in training a Deep Learning (DL) Keras model using blockchain technologies in a decentralised way. This test explores the feasibility of training a single ML model using different simulation datasets coming from different research groups. In this work, we also discuss a framework that instead makes different ML models compete against each other on the same dataset. The final goal is then to train the most performant ML model across the entire scientific community for a given experiment, either using all of the datasets available or selecting the model which performs best among every model developed in the community.
Encrypted distributed state estimation via affine averaging

Distributed state estimation arises in many applications such as position estimation in robot swarms, clock synchronization for processor networks, and data fusion. One characteristic is that agents only have access to noisy measurements of deviations between their own and neighboring states. Still, estimations of their actual state can be obtained in a fully distributed manner using algorithms such as affine averaging. However, running this algorithm, requires that the agents exchange their current state estimations, which can be a privacy issue (since they eventually reveal the actual states). To counteract this threat, we propose an encrypted version of the affine averaging algorithm in this paper. More precisely, we use homomorphic encryption to realize an encrypted implementation, where only one ``leader'' agent has access to its state estimation in plaintext. One main challenge (which often arises for recursive encrypted computations) is to prevent overflow w.r.t.~the bounded message space of the cryptosystem. We solve this problem by periodically resetting the agents' states with the help of the leader. We study the resulting system dynamics with respect to different reset strategies and support our findings with extensive numerical simulations.