Reviewerly/exHarmony huggingface dataset:
Anonymizing Machine Learning Models

There is a known tension between the need to analyze personal data to drive business and the need to preserve the privacy of data subjects. Many data protection regulations, including the EU General Data Protection Regulation (GDPR) and the California Consumer Protection Act (CCPA), set out strict restrictions and obligations on the collection and processing of personal data. Moreover, machine learning models themselves can be used to derive personal information, as demonstrated by recent membership and attribute inference attacks. Anonymized data, however, is exempt from the obligations set out in these regulations. It is therefore desirable to be able to create models that are anonymized, thus also exempting them from those obligations, in addition to providing better protection against attacks. Learning on anonymized data typically results in significant degradation in accuracy. In this work, we propose a method that is able to achieve better model accuracy by using the knowledge encoded within the trained model, and guiding our anonymization process to minimize the impact on the model’s accuracy, a process we call accuracy-guided anonymization. We demonstrate that by focusing on the model’s accuracy rather than generic information loss measures, our method outperforms state of the art k-anonymity methods in terms of the achieved utility, in particular with high values of k and large numbers of quasi-identifiers. We also demonstrate that our approach has a similar, and sometimes even better ability to prevent membership inference attacks as approaches based on differential privacy, while averting some of their drawbacks such as complexity, performance overhead and model-specific implementations. In addition, since our approach does not rely on making modifications to the training algorithm, it can even work with “black-box” models where the data owner does not have full control over the training process, or within complex machine learning pipelines where it may be difficult to replace existing learning algorithms with new ones. This makes model-guided anonymization a legitimate substitute for such methods and a practical approach to creating privacy-preserving models.
Korean Student Perceptions of a CANVAS Based EFL Class During COVID-19 －A Case Study

This study was designed to examine the feelings and perceptions of English learners at a Korean university towards using CANVAS as an online teaching medium during the global COVID-19 pandemic. Data were collected from 129 students enrolled at a university in the greater Seoul area. A survey questionnaire consisting of a combination of multiple choice, Likert scale and open-ended questions was used as the primary data collection method. In addition, if consent was given, survey participants were contacted by e-mail to discuss responses in further detail. Data was analyzed using general qualitative analysis and discussed using descriptive statistics. The findings show that despite pandemic related disruption and some feelings of frustration related to the efficiency of the system, students were able to adapt quickly to using CANVAS as their main learning interface and were generally satisfied with it. In addition, they relied on a specific number of CANVAS functions which helped with the learning process and improvement of English ability. Administrational and educational implications of these findings as well as limitations are also discussed.
Memory Allocation for Neural Networks using Graph Coloring

The Memory Allocation problem for neural networks can be represented as a two-dimensional optimization problem. The neural network is allocated into limited memory space while allocating as much data as possible into the low latency memory. Our solution is based on a generalization of graph coloring, edge-to-node transformation and considers the order in which the graph nodes are colored. We observed improvement of more than 40% in SRAM memory bandwidth in various neural networks.
On perfectly secure 2PC in the OT-hybrid model

A well known result by Kilian (1988) [23] asserts that general secure two computation (2PC) with statistical security, can be based on OT. Specifically, in the client-server model, where only one party – the client – receives an output, Kilian's result shows that given the ability to call an ideal oracle that computes OT, two parties can securely compute an arbitrary function of their inputs with unconditional security. Ishai et al. (2011) [20] further showed that this can be done efficiently for every two-party functionality in NC1 in a single round. However, their results only achieve statistical security, namely, it is allowed to have some error in security. This leaves open the natural question as to which client-server functionalities can be computed with perfect security in the OT-hybrid model, and what is the round complexity of such computation. So far, only a handful of functionalities were known to have such protocols. In addition to the obvious theoretical appeal of the question towards better understanding secure computation, perfect, as opposed to statistical reductions, may be useful for designing secure multiparty protocols with high concrete efficiency, achieved by eliminating the dependence on a security parameter. In this work, we identify a large class of client-server functionalities f:X×Y↦{0,1}, where the server's domain X is larger than the client's domain Y, that have a perfect reduction to OT. Furthermore, our reduction is 1-round using an oracle to secure evaluation of many parallel invocations of 1-out-of-2 bit OT, as done by Ishai et al. (2011) [20]. Interestingly, the set of functions that we are able to compute was previously identified by Asharov (2014) [3] in the context of fairness in two-party computation, naming these functions full-dimensional. Our result also extends to randomized non-Boolean functions f:X×Y↦{0,…,k−1} satisfying |X|>(k−1)⋅|Y|.
Benchmarking Many-Objective Topology and Weight Evolution of Neural Networks: A Study with NEWS/D

This study aims to provide procedures and benchmark problems to test the optimization capabilities of algorithms for Many-Objective Topology and Weight Evolution of Artificial Neural Networks (MaO-TWEANNs). In particular, the proposed benchmarks are based on a combination of continuous functions that have commonly been used to test single-objective optimization algorithms. In addition, this paper applies the proposed procedures and benchmark problems to evaluate NEWS/D, which is such an algorithm that has recently been introduced. The results of this study validate the optimization capabilities of NEWS/D for MaO-TWEANN problems with continuous outputs.
Pose Determination of the Disc Cutter Holder of Shield Machine Based on Monocular Vision

The visual measurement system plays a vital role in the disc cutter changing robot of the shield machine, and its accuracy directly determines the success rate of the disc cutter grasping. However, the actual industrial environment with strong noise brings a great challenge to the pose measurement methods. The existing methods are difficult to meet the required accuracy of pose measurement based on machine vision under the disc cutter changing conditions. To solve this problem, we propose a monocular visual pose measurement method consisting of the high precision optimal solution to the PnP problem (OPnP) method and the highly robust distance matching (DM) method. First, the OPnP method is used to calculate the rough pose of the shield machine's cutter holder, and then the DM method is used to measure its pose accurately. Simulation results show that the proposed monocular measurement method has better accuracy and robustness than the several mainstream PnP methods. The experimental results also show that the maximum error of the proposed method is 0.28° in the direction of rotation and 0.32 mm in the direction of translation, which can meet the measurement accuracy requirement of the vision system of the disc cutter changing robot in practical engineering application.
Hateful Memes Classification using Machine Learning

Several studies produced sophisticated models for sentiment analysis of textual data, and many others tackled feature extraction from images. However, far fewer studies focus on the multimodal representation of data, namely the information that consists of multiple channels. In this work, we focus on the classification problem of multimodal data. Memes comprise a visual image and a textual caption. This work is dedicated to classifying hateful memes and this work proposes two approaches to solve the multimodal classification problem. First, converting the visual channel into a textual one and feed it to textual classifiers. The other approach, which yielded superior results, converted both channels into a vector representation and then combined them to represent the visual-textual context. This work is a consequence of the Facebook Hateful Memes challenge. The model developed in this work managed to rank 32 among 3172 competitors in the challenge. The model is implemented with no domain knowledge or understanding of hate speech. This model performed well in the Facebook Hateful Memes challenge dataset and a novel dataset that we created to prove the consistency of generic models over other models that are structured according to domain knowledge. In contrast to the top solution in the Facebook Memes Challenge, this work provides a generic approach, without hard-coding rules ahead of training or validation, that is able to learn the hatefulness definition from any dataset. A novel dataset that comprises hateful memes retrieved randomly from the web is described in this work, which is used as another dataset to test approaches generality.
On the Evolution of the MCTS Upper Confidence Bounds for Trees by Means of Evolutionary Algorithms in the Game of Carcassonne

Monte Carlo Tree Search (MCTS) is a sampling best-first method to search for optimal decisions. The MCTS’ popularity is based on its extraordinary results in the challenging two-player based game Go, a game considered much harder than Chess and that until very recently was considered infeasible for Artificial Intelligence methods. The success of MCTS depends heavily on how the tree is built and the selection process plays a fundamental role in this. One particular selection mechanism that has proved to be reliable is based on the Upper Confidence Bounds for Trees, commonly referred as UCT. The UCT attempts to nicely balance exploration and exploitation by considering the values stored in the statistical tree of the MCTS. However, some tuning of the MCTS UCT is necessary for this to work well. In this work, we use Evolutionary Algorithms (EAs) to evolve mathematical expressions with the goal to substitute the UCT mathematical expression. We compare our proposed approach, called Evolution Strategy in MCTS (ES-MCTS) against five variants of the MCTS UCT, three variants of the *-minimax family of algorithms as well as a random controller in the Game of Carcassonne. We also use a variant of our proposed EA-based controller, dubbed ES partially integrated in MCTS. We show how the ES-MCTS controller, is able to outperform all these 10 intelligent controllers, including robust MCTS UCT controllers.
Twin Based Continuous Patching To Minimize Cyber Risk

Abstract Digital twins are virtual replicas to simulate the behavior of physical devices before they are built and to support their maintenance. We extend this technology to cybersecurity and integrate it with adversary emulation to define a remediation policy that selects and schedules patches for the vulnerabilities of an information and communication infrastructure before threat actors can exploit them. Distinct twins model, respectively, the infrastructure and threat actors. The former twin describes the infrastructure modules, their vulnerabilities, and the elementary attacks actors can implement. The attributes of the twin of a threat actor describe its attack surface, its goals, how it selects attacks, and it handles attack failures. The Haruspex software platform builds the twins of the infrastructure and those of the threat actors, and it automates the emulation of an actor. In this way, it can discover the attack paths the actor implements without disturbing the infrastructure. In each path, the actor composes elementary attacks to reach its goal. Multiple emulations can discover all the paths of an actor by covering stochastic factors such as attack success or failure. The knowledge of these paths enables the remediation policy to minimize the patches to deploy. Since new vulnerabilities continuously become public, new countermeasures are needed. A twin-based approach supports a continuous remediation process to handle changes in the infrastructure, new vulnerabilities, and new threat actors because the platform can update the twins and run adversary emulations. If new attack paths exist, the platform applies the remediation policy. Experimental data confirm the effectiveness of this approach.
Operation-level Progressive Differentiable Architecture Search

Differentiable Neural Architecture Search (DARTS) is becoming more and more popular among Neural Architecture Search (NAS) methods because of its high search efficiency and low compute cost. However, the stability of DARTS is very inferior, especially skip connections aggregation that leads to performance collapse. Though existing methods leverage Hessian eigenvalues to alleviate skip connections aggregation, they make DARTS unable to explore architectures with better performance. In the paper, we propose operation-level progressive differentiable neural architecture search (OPP-DARTS) to avoid skip connections aggregation and explore better architectures simultaneously. We first divide the search process into several stages during the search phase and increase candidate operations into the search space progressively at the beginning of each stage. It can effectively alleviate the unfair competition between operations during the search phase of DARTS by offsetting the inherent unfair advantage of the skip connection over other operations. Besides, to keep the competition between operations relatively fair and select the operation from the candidate operations set that makes training loss of the supernet largest. The experiment results indicate that our method is effective and efficient. Our method's performance on CIFAR-10 is superior to the architecture found by standard DARTS, and the transferability of our method also surpasses standard DARTS. We further demonstrate the robustness of our method on three simple search spaces, i.e., S2, S3, S4, and the results show us that our method is more robust than standard DARTS. Our code is available at https://github.com/zxunyu/OPP-DARTS.
Sufficient dimension reduction for clustered data via finite mixture modelling

Summary Sufficient dimension reduction (SDR) is an attractive approach to regression modelling. However, despite its rich literature and growing popularity in application, surprisingly little research has been done on how to perform SDR for clustered data, for example as is commonly arises in longitudinal studies. Indeed, current popular SDR methods have been mostly based on a marginal estimating equation approach. In this article, we propose a new approach to SDR for clustered data based on a combination of finite mixture modelling and mixed effects regression. Finite mixture models offer a flexible means of estimating the fixed effects central subspace, based on slicing the space up and probabilistically clustering observations to each slice (mixture component). Dimension reduction is achieved by having the mixing proportions vary only through the sufficient fixed effect predictors. We then incorporate random effects as a natural means of accounting for correlations within clusters. We employ a Monte Carlo expectation–maximisation algorithm to estimate the model parameters and fixed effects central subspace, and discuss methods for associated uncertainty quantification and prediction. Simulation studies demonstrate that our approach performs strongly against both estimating equation methods for estimating the fixed effects central subspace, and SDR methods which do not account for within‐cluster correlation. Finally, we apply the proposed approach to a data set on air pollutant monitoring across 13 stations in the Eastern United States.
Hierarchical Cooperative Swarm Policy Learning with Role Emergence

Swarm systems can cooperatively and efficiently accomplish specified complex tasks. Recent works have shown the potential of multi-agent reinforcement learning methods to study behavior policies of swarm systems. However, it is difficult for them to complete complex swarm tasks efficiently. In human society, role assignment can effectively help humans understand complex tasks and decompose them into simple certain subtasks. Inspired by this, we propose a two-level hierarchical cooperative swarm policy learning framework with role emergence based on hierarchical deep reinforcement learning for distributed swarm systems. In this framework, roles are dynamic and emergent. Agents with the same role tend to collectively complete a certain subtask. Specifically, each agent uses a higher-level swarm policy to dynamically select a role for itself in a role space and at a higher temporal scale, while it uses a lower-level swarm policy to perform the responsibilities of the selected role in a primitive action space. Meanwhile, hierarchical swarm policies with partial observation are centrally trained and decentrally executed, where agents' local interaction modules and extrinsic team rewards are designed to promote cooperation among agents. In addition, an intrinsic reward is defined to enable different roles to be identified by agents' longer-term behaviors, which implicitly associates the roles with responsibilities. Simulation results show that our method can learn and generate emergent, dynamic and identifiable roles, which helps swarm systems to reliably and efficiently accomplish complex tasks in a shorter time.
A Holey Perspective on Venn Diagrams

When interpreting the meanings of visual features in information visualizations, observers have expectations about how visual features map onto concepts (inferred mappings.) In this study, we examined whether aspects of inferred mappings that have been previously identified for colormap data visualizations generalize to a different type of visualization, Venn diagrams. Venn diagrams offer an interesting test case because empirical evidence about the nature of inferred mappings for colormaps suggests that established conventions for Venn diagrams are counterintuitive. Venn diagrams represent classes using overlapping circles and express logical relationships between those classes by shading out regions to encode the concept of non-existence, or none. We propose that people do not simply expect shading to signify non-existence, but rather they expect regions that appear as holes to signify non-existence (the hole hypothesis.) The appearance of a hole depends on perceptual properties in the diagram in relation to its background. Across three experiments, results supported the hole hypothesis, underscoring the importance of configural processing for interpreting the meanings of visual features in information visualizations.
Extended Learning Optimality Theory for the XCS classifier system on Multiple Reward Scheme

A restriction of the latest learning optimality theory for the XCS classifier system is that it targets a binary reward scheme, which virtually covers classification domains only. This paper presents an extended learning optimality theory on a multiple reward scheme, where XCS receives a reward sampled from a finite set of multiple rewards. Thus, the multiple reward scheme can be observed in various machine learning problem domains, e.g., prediction, regression, and classification. Our theory firstly proves that XCS can distinguish maximally accurate rules from any other ones with the minimum update times on the multiple-reward scheme. This proof returns theoretically optimal settings of XCS parameters, i.e., a learning rate, an error tolerance, and a subsumption threshold. In addition, suppose a continuous reward scheme as having an extremely large set of discrete rewards, our theory can be substantially satisfied on this scheme.
Detecting Contradictions in German Text: A Comparative Study

This study presents a comparison on Natural Language Inference (NLI), specifically detecting contradictions, in German text. To that end, four state-of-the-art model paradigms are being compared with respect to their performance on a machine-translated version of the well-known Stanford Natural Language Inference data set (SNLI), as well as on the German test split of the Cross-Lingual NLI corpus (XNLI). One main focus is the assessment of whether the models are robust with respect to the choice of data, and could possibly also be applied in a real-world scenario. XLM-RoBERTa outperforms the other models significantly, most likely due to its extensive pre-training and multi-head attention layers. Still, the models do not generalize very well to the XNLI data, indicating that the training corpus is too limited in topics and contradiction types. We plan to address this issue in our future work.
BlockIoT-RETEL: Blockchain and IoT Based Read-Execute-Transact-Erase-Loop Environment for Integrating Personal Health Data

The development and adoption of Electronic Health Records and health monitoring Internet of Things (IoT) devices have enabled the digitization of patient records. They have also substantially transformed the healthcare delivery system in remote patient monitoring, healthcare decision-making, and medical research. However, data tends to be fragmented among health information systems, preventing interoperability of medical data at the point of care. In order to address this gap, we propose and implement a novel architecture that utilizes blockchain and IoT technologies for integrating personal health data. The architecture comprises of two components: (1) BlockIoT contains a suite of smart contracts that define data collection and integration functionalities, and (2) RETEL utilizes a novel smart contract deployment and execution architecture. The BlockIoT-RETEL architecture combines these two components to provide a scalable mechanism for bootstrapping a personal health data collection infrastructure in the context of healthcare. We have compared the key functionalities of our system with existing comparable state-of-the-art systems in healthcare and evaluated the performance in prototypical healthcare settings with simulated data that shows promising results.