Reviewerly/exHarmony huggingface dataset:
Towards Zero-Shot Knowledge Distillation for Natural Language Processing

Knowledge distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions. In its regular manifestations, KD requires access to the teacher’s training data for knowledge transfer to the student network. However, privacy concerns, data regulations and proprietary reasons may prevent access to such data. We present, to the best of our knowledge, the first work on Zero-shot Knowledge Distillation for NLP, where the student learns from the much larger teacher without any task specific data. Our solution combines out-of-domain data and adversarial training to learn the teacher’s output distribution. We investigate six tasks from the GLUE benchmark and demonstrate that we can achieve between 75% and 92% of the teacher’s classification score (accuracy or F1) while compressing the model 30 times.
Effects of Customer Compensation on Justice Perceptions: A Longitudinal Study of Target’s Data Breach

Data breaches are a major threat to organizations from both financial and customer relations perspectives. We developed a nomological network linking post-breach compensation strategies to key outcomes, namely continued shopping intentions, positive word-of-mouth, and online complaining, with the effects being mediated by customers’ justice perceptions. We conducted a longitudinal field study investigating Target’s data breach in 2013 that affected more than 110 million customers. We examined customers’ expectations toward compensation immediately after the breach was confirmed (survey 1) and their experiences after reparations were made (survey 2). Evidence from polynomial regression and response surface analyses of data collected from 388 affected customers showed that customers’ justice perceptions were influenced by the actual compensation provided as well as the type and extent of compensation an organization could and should have provided, i.e., customers’ compensation expectations. Interestingly, both positive and negative expectation disconfirmation led to less favorable justice perceptions compared to when expectations were met. Justice perceptions were, in turn, associated with key outcomes. We discuss implications for research on data security, information systems, and justice theory.
A Self-Supervised Tibetan-Chinese Vocabulary Alignment Method

Tibetan is a low-resource language. In order to alleviate the shortage of parallel corpus between Tibetan and Chinese, this paper uses two monolingual corpora and a small number of seed dictionaries to learn the semi-supervised method with seed dictionaries and self-supervised adversarial training method through the similarity calculation of word clusters in different embedded spaces and puts forward an improved selfsupervised adversarial learning method of Tibetan and Chinese monolingual data alignment only. The experimental results are as follows. The seed dictionary of semi-supervised method made before 10 predicted word accuracy of 66.5 (Tibetan - Chinese) and 74.8 (Chinese - Tibetan) results, to improve the self-supervision methods in both language directions have reached 53.5 accuracy.
Submission to the Vaccine related fraud and security risks inquiry

This submission puts forward observations and findings from my collective research into fraud (and related cybercrime) victimisation. It draws on my knowledge and communications with scholars, law enforcement and industry bodies spanning Australia and overseas. Further, it draws specifically on the arguments I have made in the following articles: • Cross, C. (2020) Theorising the impact of COVID-19 on the fraud victimisation of older persons. Journal of Adult Protection. Online first DOI: 10.1108/JAP-08-2020-0035. • Cross, C. (2020) Fake COVID-19 testing kits and lockdown puppy scams: how to protect yourself from fraud in a pandemic. The Conversation August (19). The submission will collectively focus on these specific terms of reference: a) Telecommunications and internet fraud relating to COVID vaccinations; b) Criminal activity around the supply of fake vaccines, black market vaccines and/or fake vaccine certifications and the acquisition of certificates; c) Risks to Australia regarding fraud and integrity of COVID vaccines in South Pacific nations and support for these nations to address issues relating to fraud and integrity risks; f) Any related matters. However, prior to discussing the issue of fraud as it relates to COVID-19 vaccines as listed above, it is important to look at the broader context of fraud and how it has manifested itself during the pandemic to date.
Knowledge Distillation with Noisy Labels for Natural Language Understanding.

Knowledge Distillation (KD) is extensively used to compress and deploy large pre-trained language models on edge devices for real-world applications. However, one neglected area of research is the impact of noisy (corrupted) labels on KD. We present, to the best of our knowledge, the first study on KD with noisy labels in Natural Language Understanding (NLU). We document the scope of the problem and present two methods to mitigate the impact of label noise. Experiments on the GLUE benchmark show that our methods are effective even under high noise levels. Nevertheless, our results indicate that more research is necessary to cope with label noise under the KD.
Bar Code Labels

American Bar Codes, Inc. developed special bar code labels for inventory control of space shuttle parts and other space system components. ABC labels are made in a company-developed anodizing aluminum process and consecutively marketed with bar code symbology and human readable numbers. They offer extreme abrasion resistance and indefinite resistance to ultraviolet radiation, capable of withstanding 700 degree temperatures without deterioration and up to 1400 degrees with special designs. They offer high resistance to salt spray, cleaning fluids and mild acids. ABC is now producing these bar code labels commercially or industrial customers who also need labels to resist harsh environments.
An efficient data hiding technique in image using binary Hamming code along with particle swarm optimisation

An effective spatial domain steganographic scheme is proposed in this paper. Particle swarm optimisation (PSO) can be used to identify the most efficient pixel position in the greyscale cover image...
OPTIMAL QUANTIZATION FOR MIXED DISTRIBUTIONS

The basic goal of quantization for probability distribution is to reduce the number of values, which is typically uncountable, describing a probability distribution to some finite set and thus approximation of a continuous probability distribution by a discrete distribution. Mixed distributions are an exciting new area for optimal quantization. In this paper, we have determined the optimal sets of n-means, the nth quantization errors, and the quantization dimensions of different mixed distributions. Besides, we have discussed whether the quantization coefficients for the mixed distributions exist. The results in this paper will give a motivation and insight into more general problems in quantization for mixed distributions.
Improved Multilingual Language Model Pretraining for Social Media Text via Translation Pair Prediction

We evaluate a simple approach to improving zero-shot multilingual transfer of mBERT on social media corpus by adding a pretraining task called translation pair prediction (TPP), which predicts whether a pair of cross-lingual texts are a valid translation. Our approach assumes access to translations (exact or approximate) between source-target language pairs, where we fine-tune a model on source language task data and evaluate the model in the target language. In particular, we focus on language pairs where transfer learning is difficult for mBERT: those where source and target languages are different in script, vocabulary, and linguistic typology. We show improvements from TPP pretraining over mBERT alone in zero-shot transfer from English to Hindi, Arabic, and Japanese on two social media tasks: NER (a 37% average relative improvement in F1 across target languages) and sentiment classification (12% relative improvement in F1) on social media text, while also benchmarking on a non-social media task of Universal Dependency POS tagging (6.7% relative improvement in accuracy). Our results are promising given the lack of social media bitext corpus. Our code can be found at: https://github.com/twitter-research/multilingual-alignment-tpp.
Indústria 4.0

This article discusses the industry 4.0 revolution, taking into account the emergence and adoption of new technologies in the workplace. The research aimed to analyze and identify what the perception of the students of a College of higher education in Itabira-MG, as the influence of new intelligent machines in the workplace, your relationship with the adaptation and the emergence of new professions and skills. As to the methodological aspects, this research-oriented approach was used, quantitative data collection was carried out through the application of a structured questionnaire applied with 272 respondents with the universe characterized a census. The information was collected and processed through descriptive statistics by providing the results in this research. It was found with the results, that most of the respondents agree with the need to understand about the Fourth Industrial Revolution, technology developments in the labour market and in the workstations. As regards innovation and implementation of Artificial Intelligence (AI) of the companies, the students agree that the new era of the industry will lead to a greater competitiveness of the labor market and as a facilitator of the productive process. The data show the importance of preparing people for the job market with soft skills of the new professional profile, as well as a strong trend in developing skills for Industry 4.0 and better performance of organizational processes.
USING WHATSAPP GROUP AS AN INTERACTIVE LEARNING MEDIA IN TEACHING SYNTAX OF ENGLISH STUDENTS DEPARTMENT IAIN TULUNGAGUNG

ABSTRACT Akhbaroh, Nazliyatul. Registered Number 17203163179. Using Whatsapp Group As An Interactive Learning Media In Teaching Syntax Of English Students Department Iain Tulungagung. Advisor: Dr. Dwi Astuti Wahyu Nurhayati, SS.M.Pd.Keywords: WhatsApp, Interactive Learning Media, SyntaxThis research was conducted because of the phenomenon of online learning in the pandemic outbreak situation. The students felt the impact of this situation, including the English students of IAIN Tulungagung. The learning process had to be conducted in online learning. Most of the online class in English Department IAIN Tulungagung used WhatsApp application, like as English Syntax. Based on the pre-observation, the online class of Syntax, it became the interactive class one. So, the researcher excited to know the using of Whatsapp group as an interactive learning media in teaching syntax of English Students Department IAIN Tulungagung. The research problems of this research are: (1) how does the teacher create an interactive teaching and learning Syntax through the use of WhatsApp group? (2) how do the students perceive the use of WhatsApp group during the process of teaching and learning Syntax? (3) why is WhatsApp indicated as an interactive learning media?Research method of this study was using qualitative research design, especially in descriptive qualitative. The fourth English students academic year 2019/2020 of A class was choosen to be observed. It consists of 39 students. Research instrument of this study were interview, questionnaire in the form of Google form, and documentation to know the using WhatsApp as an interactive learning media in teaching Syntax. Based on the collecting data, the results of this study were: (1) there was question and answer section in main-activity that made the interaction each others; they could participate in the class, so the class was active, responsive. (2) Overall the using of WhatsApp group in learning Syntax got the positive perception by the students. They felt could share their opinion, interact each other, and they were not shy if they wanted ask. (3) The factors that showed WhatsApp as an interactive learning media were: it was interactive, responsive, controllable, appearing the participation, used independently, convergent features.This study is expected to give the scincetific information for the lecturer or the teacher to use this media because WhatsApp can be as an interactive learning media for online class, but the lecturer also concern to give the rules and activities that make the students more excited and also for the future researcher, this study can be as a refrence to conduct the study related to this study.
Virtual Machines on ESXi

While many readers may be looking forward to finding out how to create virtual machines (VMs) – which we will cover in this chapter – this may be the most unexciting chapter in this book, but for good reason: VMware has spent considerable time and effort to make creating VMs as easy as installing an OS on a physical machine. Yes, there are few nuances that you need to be aware of, and you will need to make sure that you get the correct ISOs for Arm processors, but in general the process of creating a VM on ESXi is straightforward.
DRIVE: One-bit Distributed Mean Estimation

We consider the problem where $n$ clients transmit $d$-dimensional real-valued vectors using $d(1+o(1))$ bits each, in a manner that allows the receiver to approximately reconstruct their mean. Such compression problems naturally arise in distributed and federated learning. We provide novel mathematical results and derive computationally efficient algorithms that are more accurate than previous compression techniques. We evaluate our methods on a collection of distributed and federated learning tasks, using a variety of datasets, and show a consistent improvement over the state of the art.
Privacy Disclosure in the Real World

Privacy protection is a hot topic in network security, many scholars are committed to evaluating privacy information disclosure by quantifying privacy, thereby protecting privacy and preventing telecommunications fraud. However, in the process of quantitative privacy, few people consider the reasoning relationship between privacy information, which leads to the underestimation of privacy disclosure and privacy disclosure caused by malicious reasoning. This paper completes an experiment on privacy information disclosure in the real world based on WordNet ontology .According to a privacy measurement algorithm, this experiment calculates the privacy disclosure of public figures in different fields, and conducts horizontal and vertical analysis to obtain different privacy disclosure characteristics. The experiment not only shows the situation of privacy disclosure, but also gives suggestions and method to reduce privacy disclosure.
Learning Rates for Nonconvex Pairwise Learning

Pairwise learning is receiving increasing attention since it covers many important machine learning tasks, e.g., metric learning, AUC maximization, and ranking. Investigating the generalization behavior of pairwise learning is thus of great significance. However, existing generalization analysis mainly focuses on the convex objective functions, leaving the nonconvex pairwise learning far less explored. Moreover, the current learning rates of pairwise learning are mostly of slower order. Motivated by these problems, we study the generalization performance of nonconvex pairwise learning and provide improved learning rates. Specifically, we develop different uniform convergence of gradients for pairwise learning under different assumptions, based on which we characterize empirical risk minimizer, gradient descent, and stochastic gradient descent. We first establish learning rates for these algorithms in a general nonconvex setting, where the analysis sheds insights on the trade-off between optimization and generalization and the role of early-stopping. We then derive faster learning rates of order <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(1/n)$</tex-math></inline-formula> for nonconvex pairwise learning with a gradient dominance curvature condition, where <inline-formula><tex-math notation="LaTeX">$n$</tex-math></inline-formula> is the sample size. Provided that the optimal population risk is small, we further improve the learning rates to <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(1/n^{2})$</tex-math></inline-formula> , which, to the best of our knowledge, are the first <inline-formula><tex-math notation="LaTeX">$\mathcal {O}(1/n^{2})$</tex-math></inline-formula> rates for pairwise learning.
Designing Smart Healthcare Systems Using Fuzzy Relations

The investment in the clinic and hospitals environments while becoming necessary very attractive, it must be justifiable with obvious level of reliability based on logical and systematically understandable steps. In this paper, we present a simple, systematic, and objective fuzzy relation-based approach for the design of a smart healthcare system. The proposed approach establishes fuzzy relations among three model building blocks: the smart enabling technologies, the healthcare system smartness features, and the healthcare system operational objectives that are desirable to realize. The max-min composition operator is utilized for combining the aforementioned relations for attaining the target relation among the enabling technologies and the operational objectives. A priority of the smart enabling technology is computed based on the total impact relation of each enabling technology on all the operational objectives. Then, the design of the smart healthcare system is reached by adopting the enabling technologies in order of priority.
Heavy Tails in SGD and Compressibility of Overparametrized Neural Networks.

Neural network compression techniques have become increasingly popular as they can drastically reduce the storage and computation requirements for very large networks. Recent empirical studies have illustrated that even simple pruning strategies can be surprisingly effective, and several theoretical studies have shown that compressible networks (in specific senses) should achieve a low generalization error. Yet, a theoretical characterization of the underlying cause that makes the networks amenable to such simple compression schemes is still missing. In this study, we address this fundamental question and reveal that the dynamics of the training algorithm has a key role in obtaining such compressible networks. Focusing our attention on stochastic gradient descent (SGD), our main contribution is to link compressibility to two recently established properties of SGD: (i) as the network size goes to infinity, the system can converge to a mean-field limit, where the network weights behave independently, (ii) for a large step-size/batch-size ratio, the SGD iterates can converge to a heavy-tailed stationary distribution. In the case where these two phenomena occur simultaneously, we prove that the networks are guaranteed to be '$\ell_p$-compressible', and the compression errors of different pruning techniques (magnitude, singular value, or node pruning) become arbitrarily small as the network size increases. We further prove generalization bounds adapted to our theoretical framework, which indeed confirm that the generalization error will be lower for more compressible networks. Our theory and numerical study on various neural networks show that large step-size/batch-size ratios introduce heavy-tails, which, in combination with overparametrization, result in compressibility.
SCULPTURE AND CARVING ART VIRTUAL MODULE BASED ON 3D AUGMENTED REALITY

Currently, online and independent learning could harm students' understanding about the shape of an object due to students’ misconceptions. This statement happens because the cognitive level possessed by each student is different. Educators who are pioneers of the main facilitators of learning in the classroom tend to take resources without considering the results of students' understanding. In sculpture and carving art material, the teacher must download 2D images from the internet, add text, and not touch objects that can provide direct tutorials. This statement is undoubtedly a problem in learning because it is related to misconceptions. In connection with these emergency problems, a module that can display 3D augmented reality objects with audio-visuals is urgently needed to avoid learning misconceptions. This virtual module can be used with independent learning methods or group discussion forums. The teacher will be a facilitator for each basic competence. The module will direct students to follow a structured and interactive learning flow. This development research uses the ADDIE Model method because the instruments and objectives are aligned with the problems on which this product is developed. This development research aims to realize learning media to avoid misconceptions and sculpt/carve following structured and directed learning stages and create expert-validated learning media, evidenced by the validation of media experts 89% and material expert validation 93%.
A Natural-Language-Processing-Based Procedure for Generating Distractors for Multiple-Choice Questions

One of the most challenging aspects of writing multiple-choice test questions is identifying plausible incorrect response options—i.e., distractors. To help with this task, a procedure is introduced that can mine existing item banks for potential distractors by considering the similarities between a new item’s stem and answer and the stems and response options for items in the bank. This approach uses natural language processing to measure similarity and requires a substantial pool of items for constructing the generating model. The procedure is demonstrated with data from the United States Medical Licensing Examination (USMLE®). For about half the items in the study, at least one of the top three system-produced candidates matched a human-produced distractor exactly; and for about one quarter of the items, two of the top three candidates matched human-produced distractors. A study was conducted in which a sample of system-produced candidates were shown to 10 experienced item writers. Overall, participants thought about 81% of the candidates were on topic and 56% would help human item writers with the task of writing distractors.