Reviewerly/exHarmony huggingface dataset:
Fuzzy-Based Kernelized Clustering Algorithms for Handling Big Data Using Apache Spark

In this paper, we propose a novel Kernelized Scalable Random Sampling with Iterative Optimization Fuzzy c-Means (KSRSIO-FCM) and a Kernelized Scalable Literal Fuzzy c-Means (KSLFCM) clustering algorithms for big data framework. The evolution of kernelized clustering algorithms led us to deal with the nonlinear separable problems by applying kernel Radial Basis Functions (RBF) which map the input data space nonlinearly into a high-dimensional feature space. The experimental result shows that the KSRSIO-FCM algorithm achieves significant improvement in terms of F-score, Adjusted Rand Index (ARI), and Normalized Mutual Information (NMI) for Big Data. Experimentation is performed on well-known IRIS datasets to show the effectiveness of proposed KSRSIO-FCM in comparison with KSLFCM. The KSRSIO-FCM implemented on Apache Spark shows better potential for Big Data clustering.
TESA: A Task in Entity Semantic Aggregation for Abstractive Summarization

Human-written texts contain frequent generalizations and semantic aggregation of content. In a document, they may refer to a pair of named entities such as ‘London’ and ‘Paris’ with different expressions: “the major cities”, “the capital cities” and “two European cities”. Yet generation, especially, abstractive summarization systems have so far focused heavily on paraphrasing and simplifying the source content, to the exclusion of such semantic abstraction capabilities. In this paper, we present a new dataset and task aimed at the semantic aggregation of entities. TESA contains a dataset of 5.3K crowd-sourced entity aggregations of Person, Organization, and Location named entities. The aggregations are document-appropriate, meaning that they are produced by annotators to match the situational context of a given news article from the New York Times. We then build baseline models for generating aggregations given a tuple of entities and document context. We finetune on TESA an encoder-decoder language model and compare it with simpler classification methods based on linguistically informed features. Our quantitative and qualitative evaluations show reasonable performance in making a choice from a given list of expressions, but free-form expressions are understandably harder to generate and evaluate.
Model Stealing Defense with Hybrid Fuzzy Models: Work-in-Progress

With increasing applications of Deep Neural Networks (DNNs) to edge computing systems, security issues have received more attentions. Particularly, model stealing attack is one of the biggest challenge to the privacy of models. To defend against model stealing attack, we propose a novel protection architecture with fuzzy models. Each fuzzy model is designed to generate wrong predictions corresponding to a particular category. In addition' we design a special voting strategy to eliminate the systemic errors, which can destroy the dark knowledge in predictions at the same time. Preliminary experiments show that our method substantially decreases the clone model's accuracy (up to 20%) without loss of inference accuracy for benign users.
Self-Supervised Relational Reasoning for Representation Learning

In self-supervised learning, a system is tasked with achieving a surrogate objective by defining alternative targets on a set of unlabeled data. The aim is to build useful representations that can be used in downstream tasks, without costly manual annotation. In this work, we propose a novel self-supervised formulation of relational reasoning that allows a learner to bootstrap a signal from information implicit in unlabeled data. Training a relation head to discriminate how entities relate to themselves (intra-reasoning) and other entities (inter-reasoning), results in rich and descriptive representations in the underlying neural network backbone, which can be used in downstream tasks such as classification and image retrieval. We evaluate the proposed method following a rigorous experimental procedure, using standard datasets, protocols, and backbones. Self-supervised relational reasoning outperforms the best competitor in all conditions by an average 14% in accuracy, and the most recent state-of-the-art model by 3%. We link the effectiveness of the method to the maximization of a Bernoulli log-likelihood, which can be considered as a proxy for maximizing the mutual information, resulting in a more efficient objective with respect to the commonly used contrastive losses.
Confidence sequences for sampling without replacement

Many practical tasks involve sampling sequentially without replacement (WoR) from a finite population of size $N$, in an attempt to estimate some parameter $\theta^\star$. Accurately quantifying uncertainty throughout this process is a nontrivial task, but is necessary because it often determines when we stop collecting samples and confidently report a result. We present a suite of tools for designing confidence sequences (CS) for $\theta^\star$. A CS is a sequence of confidence sets $(C_n)_{n=1}^N$, that shrink in size, and all contain $\theta^\star$ simultaneously with high probability. We present a generic approach to constructing a frequentist CS using Bayesian tools, based on the fact that the ratio of a prior to the posterior at the ground truth is a martingale. We then present Hoeffding- and empirical-Bernstein-type time-uniform CSs and fixed-time confidence intervals for sampling WoR, which improve on previous bounds in the literature and explicitly quantify the benefit of WoR sampling.
Regularized linear autoencoders recover the principal components, eventually

Our understanding of learning input-output relationships with neural nets has improved rapidly in recent years, but little is known about the convergence of the underlying representations, even in the simple case of linear autoencoders (LAEs). We show that when trained with proper regularization, LAEs can directly learn the optimal representation -- ordered, axis-aligned principal components. We analyze two such regularization schemes: non-uniform $\ell_2$ regularization and a deterministic variant of nested dropout [Rippel et al, ICML' 2014]. Though both regularization schemes converge to the optimal representation, we show that this convergence is slow due to ill-conditioning that worsens with increasing latent dimension. We show that the inefficiency of learning the optimal representation is not inevitable -- we present a simple modification to the gradient descent update that greatly speeds up convergence empirically.
Sample Complexity of Uniform Convergence for Multicalibration

There is a growing interest in societal concerns in machine learning systems, especially in fairness. Multicalibration gives a comprehensive methodology to address group fairness. In this work, we address the multicalibration error and decouple it from the prediction error. The importance of decoupling the fairness metric (multicalibration) and the accuracy (prediction error) is due to the inherent trade-off between the two, and the societal decision regarding the right tradeoff (as imposed many times by regulators). Our work gives sample complexity bounds for uniform convergence guarantees of multicalibration error, which implies that regardless of the accuracy, we can guarantee that the empirical and (true) multicalibration errors are close. We emphasize that our results: (1) are more general than previous bounds, as they apply to both agnostic and realizable settings, and do not rely on a specific type of algorithm (such as deferentially private), (2) improve over previous multicalibration sample complexity bounds and (3) implies uniform convergence guarantees for the classical calibration error.
An Empirical Study of Production-oriented Approach in College English Writing Teaching

Production-oriented approach" (POA) is a foreign language teaching theory put forward by Chinese professor Wen Qiufang and her team.The theoretical system of POA consists of teaching concept, teaching hypothesis and teaching process with the role of teachers' mediation, aiming at solving the problem of "separation between learning and using" and enhancing foreign language teaching effectiveness in college.The empirical experiment has been conducted in a university in Ningxia Hui Autonomous Region for 15 weeks, and 110 sophomores of two parallel classes participate in the research with 55 students in each class.They are taught by the same teacher with the same teaching materials in both experimental class (EC) and control class (CC).The only difference is that EC is taught by POA, while CC is taught by traditional writing approach that is product writing approach (PWA).Furthermore, questionnaires and writing tests are implemented in CC and EC, as well as Statistic Package for Social Science (SPSS) 17.0 is used to collect and analyze data.The research reveals that POA is more effective to help students master English writing strategies than PWA.Meanwhile, POA is more capable to improve students' language accuracy and increase the output of targeted connectives significantly instead of syntactic complexity of clause in English writing.Regarding writing scores, there is no significant difference between CC and EC which is consistent with the research results gotten by Zhang [20].At last, some insightful suggestions and implications are proposed for future study.
MESA: Boost Ensemble Imbalanced Learning with MEta-SAmpler.

Imbalanced learning (IL), i.e., learning unbiased models from class-imbalanced data, is a challenging problem. Typical IL methods including resampling and reweighting were designed based on some heuristic assumptions. They often suffer from unstable performance, poor applicability, and high computational cost in complex tasks where their assumptions do not hold. In this paper, we introduce a novel ensemble IL framework named MESA. It adaptively resamples the training set in iterations to get multiple classifiers and forms a cascade ensemble model. MESA directly learns the sampling strategy from data to optimize the final metric beyond following random heuristics. Moreover, unlike prevailing meta-learning-based IL solutions, we decouple the model-training and meta-training in MESA by independently train the meta-sampler over task-agnostic meta-data. This makes MESA generally applicable to most of the existing learning models and the meta-sampler can be efficiently applied to new tasks. Extensive experiments on both synthetic and real-world tasks demonstrate the effectiveness, robustness, and transferability of MESA. Our code is available at this https URL.
GramGAN: Deep 3D Texture Synthesis From 2D Exemplars

We present a novel texture synthesis framework, enabling the generation of infinite, high-quality 3D textures given a 2D exemplar image. Inspired by recent advances in natural texture synthesis, we train deep neural models to generate textures by non-linearly combining learned noise frequencies. To achieve a highly realistic output conditioned on an exemplar patch, we propose a novel loss function that combines ideas from both style transfer and generative adversarial networks. In particular, we train the synthesis network to match the Gram matrices of deep features from a discriminator network. In addition, we propose two architectural concepts and an extrapolation strategy that significantly improve generalization performance. In particular, we inject both model input and condition into hidden network layers by learning to scale and bias hidden activations. Quantitative and qualitative evaluations on a diverse set of exemplars motivate our design decisions and show that our system performs superior to previous state of the art. Finally, we conduct a user study that confirms the benefits of our framework.
Model-based Reinforcement Learning for Semi-Markov Decision Processes with Neural ODEs

We present two elegant solutions for modeling continuous-time dynamics, in a novel model-based reinforcement learning (RL) framework for semi-Markov decision processes (SMDPs), using neural ordinary differential equations (ODEs). Our models accurately characterize continuous-time dynamics and enable us to develop high-performing policies using a small amount of data. We also develop a model-based approach for optimizing time schedules to reduce interaction rates with the environment while maintaining the near-optimal performance, which is not possible for model-free methods. We experimentally demonstrate the efficacy of our methods across various continuous-time domains.
Learning Deep Attribution Priors Based On Prior Knowledge

Feature attribution methods, which explain an individual prediction made by a model as a sum of attributions for each input feature, are an essential tool for understanding the behavior of complex deep learning models. However, ensuring that models produce meaningful explanations, rather than ones that rely on noise, is not straightforward. Exacerbating this problem is the fact that attribution methods do not provide insight as to why features are assigned their attribution values, leading to explanations that are difficult to interpret. In real-world problems we often have sets of additional information for each feature that are predictive of that feature's importance to the task at hand. Here, we propose the deep attribution prior (DAPr) framework to exploit such information to overcome the limitations of attribution methods. Our framework jointly learns a relationship between prior information and feature importance, as well as biases models to have explanations that rely on features predicted to be important. We find that our framework both results in networks that generalize better to out of sample data and admits new methods for interpreting model behavior.
Examples and Applications on Genetic Algorithms

The chapter discusses several examples and applications based on genetic algorithms with problem definitions, suitable encoding schemes, applications of genetic operators, and the design of fitness functions with the overall evolution process. All the examples discussed in the chapter accompany the necessary illustrations and data for a better understanding and effective representation of the problems and solutions. Initial examples in the chapter include function optimization with single and multiple variables. Here, the real-life examples such as the use of the genetic algorithm in profit and investment, maximizing the number of a digit/character in a sentence, multivariable function optimization, etc. are demonstrated with solutions besides various numerical examples. Problems that fall in the routine but expert/intelligent category such as best student selection, mobile selection (with different encodings), and car selection are also discussed in this chapter. These problems require specific encoding and application-specific fitness functions, which are discussed in detail with illustrations. Tricks and tactics of problem-solving are also described here e.g. how to solve a minimization problem that comes in the disguise of maximization.
A Novel Automated Curriculum Strategy to Solve Hard Sokoban Planning Instances

In recent years, we have witnessed tremendous progress in deep reinforcement learning (RL) for tasks such as Go, Chess, video games, and robot control. Nevertheless, other combinatorial domains, such as AI planning, still pose considerable challenges for RL approaches. The key difficulty in those domains is that a positive reward signal becomes {\em exponentially rare} as the minimal solution length increases. So, an RL approach loses its training signal. There has been promising recent progress by using a curriculum-driven learning approach that is designed to solve a single hard instance. We present a novel {\em automated} curriculum approach that dynamically selects from a pool of unlabeled training instances of varying task complexity guided by our {\em difficulty quantum momentum} strategy. We show how the smoothness of the task hardness impacts the final learning results. In particular, as the size of the instance pool increases, the ``hardness gap'' decreases, which facilitates a smoother automated curriculum based learning process. Our automated curriculum approach dramatically improves upon the previous approaches. We show our results on Sokoban, which is a traditional PSPACE-complete planning problem and presents a great challenge even for specialized solvers. Our RL agent can solve hard instances that are far out of reach for any previous state-of-the-art Sokoban solver. In particular, our approach can uncover plans that require hundreds of steps, while the best previous search methods would take many years of computing time to solve such instances. In addition, we show that we can further boost the RL performance with an intricate coupling of our automated curriculum approach with a curiosity-driven search strategy and a graph neural net representation.
Model Inversion Networks for Model-Based Optimization

In this work, we aim to solve data-driven optimization problems, where the goal is to find an input that maximizes an unknown score function given access to a dataset of inputs with corresponding scores. When the inputs are high-dimensional and valid inputs constitute a small subset of this space (e.g., valid protein sequences or valid natural images), such model-based optimization problems become exceptionally difficult, since the optimizer must avoid out-of-distribution and invalid inputs. We propose to address such problem with model inversion networks (MINs), which learn an inverse mapping from scores to inputs. MINs can scale to high-dimensional input spaces and leverage offline logged data for both contextual and non-contextual optimization problems. MINs can also handle both purely offline data sources and active data collection. We evaluate MINs on tasks from the Bayesian optimization literature, high-dimensional model-based optimization problems over images and protein designs, and contextual bandit optimization from logged data.
Dispossession, Un-freedom, Precarity

The partition of the Indian subcontinent forced millions of people to flee to the other side of the borders, freshly demarcated by the British colonial rulers just on the eve of their departure from South Asia. Almost a decade-long migration of people could not, however, settle the boundaries and lives of the people once and for all. The postcolonial rulers retained many of the draconian laws of the late colonial period, like the Foreigners’ Act in India, and laced them with new laws and regulations, thus leading to greater dispossession of people of homes, generating widespread situations of un-freedom, and creating countless refugees and stateless persons, mostly forced to survive in sites of precarious life, without any right to have rights. The concern of this contribution is this politics of dispossession in postcolonial South Asia and its relation with citizenship laws of the region.
Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight

Transfer learning can boost the performance on the target task by leveraging the knowledge of the source domain. Recent works in neural architecture search (NAS), especially one-shot NAS, can aid transfer learning by establishing sufficient network search space. However, existing NAS methods tend to approximate huge search spaces by explicitly building giant super-networks with multiple sub-paths, and discard super-network weights after a child structure is found. Both the characteristics of existing approaches causes repetitive network training on source tasks in transfer learning. To remedy the above issues, we reduce the super-network size by randomly dropping connection between network blocks while embedding a larger search space. Moreover, we reuse super-network weights to avoid redundant training by proposing a novel framework consisting of two modules, the neural architecture search module for architecture transfer and the neural weight search module for weight transfer. These two modules conduct search on the target task based on a reduced super-networks, so we only need to train once on the source task. We experiment our framework on both MS-COCO and CUB-200 for the object detection and fine-grained image classification tasks, and show promising improvements with only $$O(C^{N})$$ super-network complexity.
Knowledge Graph of Design Rules for a Context-Aware Cognitive Design Assistant

[Context] The design of a system shall comply with many design rules that help industrial designers to create high quality design in an efficient way. Nowadays, design rules try to consider all product lifecycle’s phases leading to an ever-increasing growth. This context makes the management of design rules a difficult but essential task. This is why many research and industrial works try to automate this task [1, 3, 4]. [Problem] The processing of design rules, which are natural language sentences stored in unstructured documents, requires expert software. Moreover, existing tools interrupt the design workflow and slow down the design process. [Proposition] We propose a Context-Aware Cognitive Design Assistant (CACDA) to support designers who have to satisfy some design rules among “Big Data”. First, we describe the CACDA from the user’s perspective. Second, we detail the process for modelling unstructured design rules into a computable knowledge graph that will feed the cognitive design assistant. [Future Work] Once our knowledge graph of design rules will be operational, we will concentrate on its processing to retrieve, recommend, and verify design rules. Experiments will also help to determine pros and cons of the design assistant.
Exploring VR and AR Tools for Service Design

We explore the possibilities of augmented and virtual reality (AR and VR) as co-design tools for service design. Four expert service designers, working in industry, tried out and discussed different VR and AR based tools: Head mounted display (HMD) based VR and AR, and projected environments. The findings emphasize that HMD techniques are not favored, as they isolate the wearer from the co-design situation and hinder the observation of emotions and expressions. Projection of the design context was positively commented, as supporting collaboration in co-design sessions.
Smart and/or Mobile: An Investigation Within Czech University Students

The paper illustrates the results of a survey conducted at Jan Evangelista Purkyne University, Usti nad Labem, Czech Republic, among a selected sample of students (N = 97). The main objective of the investigation was to discover how the term smart is perceived, particularly whether respondents consider the feature of mobility to be an inseparable part of the word definition. Data were collected by the printed questionnaire consisting of six open-answer questions. Respondents provided their feedback immediately, by handwriting. The collected data were processed by the method of frequency analysis; results are presented in figures and discussed. Despite that we are surrounded by numerous smart and mobile devices and technologies, the survey results did not prove our expectations, when showing that (1) students’ understanding of the term smart was not clear and deep and (2) they did not consider the feature of mobility to be an inseparable part of the term smart. Results might have been limited by the sample characteristics (N, gender, selected faculties). Findings are briefly discussed in the context of other works that are rather rare, which evokes the call for further researches in this field.