Reviewerly/exHarmony huggingface dataset:
Transferable Graph Optimizers for ML Compilers

Most compilers for machine learning (ML) frameworks need to solve many correlated optimization problems to generate efficient machine code. Current ML compilers rely on heuristics based algorithms to solve these optimization problems one at a time. However, this approach is not only hard to maintain but often leads to sub-optimal solutions especially for newer model architectures. Existing learning based approaches in the literature are sample inefficient, tackle a single optimization problem, and do not generalize to unseen graphs making them infeasible to be deployed in practice. To address these limitations, we propose an end-to-end, transferable deep reinforcement learning method for computational graph optimization (GO), based on a scalable sequential attention mechanism over an inductive graph neural network. GO generates decisions on the entire graph rather than on each individual node autoregressively, drastically speeding up the search compared to prior methods. Moreover, we propose recurrent attention layers to jointly optimize dependent graph optimization tasks and demonstrate 33%-60% speedup on three graph optimization tasks compared to TensorFlow default optimization. On a diverse set of representative graphs consisting of up to 80,000 nodes, including Inception-v3, Transformer-XL, and WaveNet, GO achieves on average 21% improvement over human experts and 18% improvement over the prior state of the art with 15x faster convergence, on a device placement task evaluated in real systems.
Need Analysis in ADDIE Model to Develop Academic Speaking Materials

Need analysis has become part of ADDIE model in developing materials for language learning. This research aims at examining the importance of need analysis as the initial focal step prior to the development of materials for Speaking for Academic Purpose. It is also investigating the reliability of ADDIE model prior to developing teaching material, specifically the ‘analyze’ stage. This research is typically a qualitative descriptive approach which conducted by using survey method. The instruments are students questionnaires and experts questionnaires created in online google form. There are totally 68 higher education students consist of thirty-three (33) seventh semester students and thirty-five (35) of fifth semester students taking the role as the respondents. The experts questionnaires are distributed to six lecturers who have the same interest in materials development and teach English for academic Purposes. The most result showed that 40 and 21 respondents consider ‘very important’ and ‘important’ which shows that 90% of all respondents has similar opinion on the importance of textbook for the course of Speaking for Academic Purpose availability. All of the expert-respondents consider that it is important to develop materials for the course of Speaking for Academic purpose, shown by the dominant answers of ‘yes’Thus, by implementing ADDIE model, the researchers are convinced to develop materials for Speaking for Academic Purpose for students of English Education Study Program in Nusa Nipa University.
Bi-tuning of Pre-trained Representations

It is common within the deep learning community to first pre-train a deep neural network from a large-scale dataset and then fine-tune the pre-trained model to a specific downstream task. Recently, both supervised and unsupervised pre-training approaches to learning representations have achieved remarkable advances, which exploit the discriminative knowledge of labels and the intrinsic structure of data, respectively. It follows natural intuition that both discriminative knowledge and intrinsic structure of the downstream task can be useful for fine-tuning, however, existing fine-tuning methods mainly leverage the former and discard the latter. A question arises: How to fully explore the intrinsic structure of data for boosting fine-tuning? In this paper, we propose Bi-tuning, a general learning framework to fine-tuning both supervised and unsupervised pre-trained representations to downstream tasks. Bi-tuning generalizes the vanilla fine-tuning by integrating two heads upon the backbone of pre-trained representations: a classifier head with an improved contrastive cross-entropy loss to better leverage the label information in an instance-contrast way, and a projector head with a newly-designed categorical contrastive learning loss to fully exploit the intrinsic structure of data in a category-consistent way. Comprehensive experiments confirm that Bi-tuning achieves state-of-the-art results for fine-tuning tasks of both supervised and unsupervised pre-trained models by large margins (e.g. 10.7\% absolute rise in accuracy on CUB in low-data regime).
MECHANIZING <i>PRINCIPIA LOGICO-METAPHYSICA</i> IN FUNCTIONAL TYPE-THEORY

Abstract Principia Logico-Metaphysica contains a foundational logical theory for metaphysics, mathematics, and the sciences. It includes a canonical development of Abstract Object Theory [AOT], a metaphysical theory (inspired by ideas of Ernst Mally, formalized by Zalta) that distinguishes between ordinary and abstract objects. This article reports on recent work in which AOT has been successfully represented and partly automated in the proof assistant system Isabelle/HOL. Initial experiments within this framework reveal a crucial but overlooked fact: a deeply-rooted and known paradox is reintroduced in AOT when the logic of complex terms is simply adjoined to AOT’s specially formulated comprehension principle for relations. This result constitutes a new and important paradox, given how much expressive and analytic power is contributed by having the two kinds of complex terms in the system. Its discovery is the highlight of our joint project and provides strong evidence for a new kind of scientific practice in philosophy, namely, computational metaphysics . Our results were made technically possible by a suitable adaptation of Benzmüller’s metalogical approach to universal reasoning by semantically embedding theories in classical higher-order logic. This approach enables one to reuse state-of-the-art higher-order proof assistants, such as Isabelle/HOL, for mechanizing and experimentally exploring challenging logics and theories such as AOT. Our results also provide a fresh perspective on the question of whether relational type theory or functional type theory better serves as a foundation for logic and metaphysics.
Universal random codes: capacity regions of the compound quantum multiple-access channel with one classical and one quantum sender

We consider the compound memoryless quantum multiple-access channel (QMAC) with two sending terminals. In this model, the transmission is governed by the memoryless extensions of a completely positive and trace preserving map which can be any element of a prescribed set of possible maps. We study a communication scenario, where one of the senders aims for transmission of classical messages, while the other sender sends quantum information. Combining powerful universal random coding results for classical and quantum information transmission over point-to-point channels, we establish universal codes for the mentioned two-sender task. Conversely, we prove that the two-dimensional rate region achievable with these codes is optimal. In consequence, we obtain a multi-letter characterization of the capacity region of each compound QMAC for the considered transmission task.
Lessons Learned While Migrating From Swing to JavaFX

We describe a case study of the migration of an interactive diagramming tool written in Java from the Swing graphical user interface framework to JavaFX. The outcome is presented as five lessons about the search for information when migrating software between major frameworks.
High Dimensional Single-Index Bayesian Modeling of Brain Atrophy

We propose a model of brain atrophy as a function of high-dimensional genetic information and low dimensional covariates such as gender, age, APOE gene, and disease status. A nonparametric single-index Bayesian model of high dimension is proposed to model the relationship with B-spline series prior on the unknown functions and Dirichlet process scale mixture of centered normal prior on the distributions of the random effects. The posterior rate of contraction without the random effect is established for a fixed number of regions and time points with increasing sample size. We implement an efficient computation algorithm through a Hamiltonian Monte Carlo (HMC) algorithm. The performance of the proposed Bayesian method is compared with the corresponding least square estimator in the linear model with horseshoe prior, LASSO and SCAD penalization on the high-dimensional covariates. The proposed Bayesian method is applied to a dataset on volumes of brain regions recorded over multiple visits of 748 individuals using 620,901 SNPs and 6 other covariates for each individual, to identify factors associated with brain atrophy.
More Embeddings, Better Sequence Labelers?

Recent work proposes a family of contextual embeddings that significantly improves the accuracy of sequence labelers over noncontextual embeddings.However, there is no definite conclusion on whether we can build better sequence labelers by combining different kinds of embeddings in various settings.In this paper, we conduct extensive experiments on 3 tasks over 18 datasets and 8 languages to study the accuracy of sequence labeling with various embedding concatenations and make three observations: (1) concatenating more embedding variants leads to better accuracy in rich-resource and cross-domain settings and some conditions of low-resource settings; (2) concatenating contextual sub-word embeddings with contextual character embeddings hurts the accuracy in extremely lowresource settings; (3) based on the conclusion of (1), concatenating additional similar contextual embeddings cannot lead to further improvements.We hope these conclusions can help people build stronger sequence labelers in various settings.
The squashed entanglement of the noiseless quantum Gaussian attenuator and amplifier

We determine the maximum squashed entanglement achievable between sender and receiver of the noiseless quantum Gaussian attenuators and amplifiers and we prove that it is achieved sending half of an infinitely squeezed two-mode vacuum state. The key ingredient of the proof is a lower bound to the squashed entanglement of the quantum Gaussian states obtained applying a two-mode squeezing operation to a quantum thermal Gaussian state tensored with the vacuum state. This is the first lower bound to the squashed entanglement of a quantum Gaussian state and opens the way to determine the squashed entanglement of all quantum Gaussian channels. Moreover, we determine the classical squashed entanglement of the quantum Gaussian states above and show that it is strictly larger than their squashed entanglement. This is the first time that the classical squashed entanglement of a mixed quantum Gaussian state is determined.
The Myth of the Importance of Consistency in Analytical Hierarchical Processes

In this work, rather than relating different areas of knowledge, the authors make use of logic, which is intrinsic to mathematical models. They discuss one of the fallacies that hides behind the analytic hierarchy process (AHP). In particular, the analysis of the consistency index, sometimes referred as the Saaty consistency or simply the consistency, will be discussed. The AHP is based on three fundamental axioms: reciprocity, homogeneity, and synthesis. However, it gives a great relevance to consistency, which is managed like a fourth axiom. On each of these four axioms, some comments will be made. However, this work will focus mainly on discussing the importance of the consistency index, to which, undoubtedly, a large number of essays and many hours of work have been devoted. In trying to answer the question, the calculation of consistency is really important. The aim of the paper is to raise a discussion of one of the fallacies that hides behind the analytic hierarchy process and present a simple and direct way to handle Saaty consistency.
Digesting Network Traffic for Forensic Investigation Using Digital Signal Processing Techniques

One of the most important practices of cybercrime investigations is to search a network traffic history for an excerpt of traffic, such as the disclosed information of an organization or a worm's signature. In post-mortem investigations, criminals and targets are detected by attributing the excerpt to payloads of traffic flows. Since it is impossible to store the high volume of data related to long-term traffic history, payload attribution systems (PASs) based on storing a compact digest of traffic using Bloom filters have been presented in the literature. In these systems, querying the stored digest for an excerpt results in a low number of suspects instead of certain criminals. In this paper, we present a different PAS which is based on simple and widespread digital signal processing techniques. Our traffic digesting scheme has been inspired by DSP-based compression algorithms. The proposed payload attribution system, named DSPAS, not only results in a low false positive rate but also outperforms previous schemes in response to wildcard queries which are essentially applicable for complex excerpts, such as the signature of polymorphic worms. Our theoretical analysis and practical evaluations show that the DSPAS results in a significantly lower false positive rate and also a lower processing time for wildcard queries in comparison to previous works. Moreover, our PAS can prevent a malicious insider from evading the PAS by its ability to find strings similar to a queried excerpt.
Learning Concepts Definable in First-Order Logic with Counting

We study classification problems over relational background structures for hypotheses that are defined using logics with counting. The aim of this paper is to find learning algorithms running in time sublinear in the size of the background structure. We show that hypotheses defined by FOCN(P)formulas over structures of polylogarithmic degree can be learned in sublinear time. Furthermore, we prove that for structures of unbounded degree there is no sublinear learning algorithm for first-order formulas.
Using Pre-Trained Transformer for Better Lay Summarization

In this paper, we tack lay summarization tasks, which aim to automatically produce lay summaries for scientific papers, to participate in the first CL-LaySumm 2020 in SDP workshop at EMNLP 2020.We present our approach of using Pre-training with Extracted Gap-sentences for Abstractive Summarization (PEGASUS; Zhang et al., 2019b) to produce the lay summary and combining those with the extractive summarization model using Bidirectional Encoder Representations from Transformers (BERT;Devlin et al., 2019) and readability metrics that measure the readability of the sentence to further improve the quality of the summary.Our model achieves a remarkable performance on ROUGE metrics, demonstrating the produced summary is more readable while it summarizes the main points of the document.
QCNN Inspired Reconfigurable Keyword Spotting Processor With Hybrid Data-Weight Reuse Methods

The keyword spotting (KWS) system is one of the most important interfaces between humans and machines since it is usually the start of automatic speech recognition and natural language processing techniques. However, for KWS hardware, it is still a problem to make one specified chip both low power and high performed under multiple scenarios, such as in meeting rooms, on different traffic or in parks and so on, for different scenarios own wide range signal-noise-ratios (SNRs). The problem leads to the requirements of balanced design between KWS system accuracy and the hardware cost under various noise types and levels. To overcome the balanced design and tradeoff problems, a complete KWS processor including an Mel-Frequency Cepstrum Coefficients (MFCC) feature extractor and a quantized Convolutional Neural Network (QCNN) accelerator is proposed for wide SNR range and low-power KWS in this paper. Firstly, the approach to quantize CNNs into QCNNs with high accuracy is proposed with considerations of hardware-software tradeoff. With the tradeoff of KWS system accuracy and hardware cost, the 4bit/8bit dual-working-mode strategy is proposed to keep low hardware cost and high accuracy under different scenarios. To be specific, the training, tuning and validating of the CNNs and QCNNs are taken with the dataset of 10 keywords chosen from the Google Command Speech Dataset (GCSD). Secondly, a serial FFT based MFCC extractor is implemented with low power and small footprint. Finally, with a novel hybrid reuse strategy of input data and network weight, a reconfigurable and approximate computing based QCNN accelerator is designed. Implemented and verified under TSMC 22nm ULL technology, with the area of 1.42mm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> , the QCNN accelerator can achieve 5.26μW/9.08μW power consumption in 4bit/8bit work mode with accuracy of 88% and 93% respectively, which is superior to the state-of-the-art processors.
Hcore-Init: Neural Network Initialization based on Graph Degeneracy

Neural networks have become a very popular tool for many machine learning tasks, as in recent years we witnessed many novel architectures, learning and optimization techniques for deep learning. Capitalizing on the fact that neural networks inherently constitute multipartite graphs among neuron layers, we aim to analyze directly their structure to extract meaningful information that can improve the learning process. To our knowledge graph mining techniques for enhancing learning in neural networks have not been thoroughly investigated. In this paper we propose an adapted version of the k-core structure for the complete weighted multipartite graph extracted from a deep learning architecture. As a multipartite graph is a combination of bipartite graphs, that are in turn the incidence graphs of hypergraphs, we design k-hypercore decomposition, the hypergraph analogue of k-core degeneracy. We applied k-hypercore to several neural network architectures, more specifically to convolutional neural networks and multilayer perceptrons for image recognition tasks after a very short pretraining. Then we used the information provided by the hypercore numbers of the neurons to re-initialize the weights of the neural network, thus biasing the gradient optimization scheme. Extensive experiments proved that k-hypercore outperforms the state-of-the-art initialization methods.
Assessing Human-Parity in Machine Translation on the Segment Level

Recent machine translation shared tasks have shown top-performing systems to tie or in some cases even outperform human translation.Such conclusions about system and human performance are, however, based on estimates aggregated from scores collected over large test sets of translations and so leave some remaining questions unanswered.For instance, simply because a system significantly outperforms the human translator on average may not necessarily mean that it has done so for every translation in the test set.Furthermore, are there remaining source segments present in evaluation test sets that cause significant challenges for top-performing systems and can such challenging segments go unnoticed due to the opacity of current human evaluation procedures?To provide insight into these issues we carefully inspect the outputs of top-performing systems in the recent WMT19 news translation shared task for all language pairs in which a system either tied or outperformed human translation.Our analysis provides a new method of identifying the remaining segments for which either machine or human perform poorly.For example, in our close inspection of WMT19 English to German and German to English we discover the segments that disjointly proved a challenge for human and machine.For English to Russian, there were no segments included in our sample of translations that caused a significant challenge for the human translator, while we again identify the set of segments that caused issues for the top-performing system.
A first look at an emerging model of community organizations for the long-term maintenance of ecosystems' packages

One of the biggest strength of many modern programming languages is their rich open source package ecosystem. Indeed, modern language-specific package managers have made it much easier to share reusable code and depend on components written by someone else (often by total strangers). However, while they make programmers more productive, such practices create new health risks at the level of the ecosystem: when a heavily-used package stops being maintained, all the projects that depend on it are threatened. In this paper, I ask three questions. RQ1: How prevalent is this threat? In particular, how many depended-upon packages are maintained by a single person (who can drop out at any time)? I show that this is the case for a significant proportion of such packages. RQ2: How can project authors that depend on a package react to its maintainer becoming unavailable? I list a few options, and I focus in particular on the notion of fork. RQ3: How can the programmers of an ecosystem react collectively to such events, or prepare for them? I give a first look at an emerging model of community organizations for the long-term maintenance of packages, that appeared in several ecosystems.
Expert-Supervised Reinforcement Learning for Offline Policy Learning and Evaluation

Offline Reinforcement Learning (RL) is a promising approach for learning optimal policies in environments where direct exploration is expensive or unfeasible. However, the adoption of such policies in practice is often challenging, as they are hard to interpret within the application context, and lack measures of uncertainty for the learned policy value and its decisions. To overcome these issues, we propose an Expert-Supervised RL (ESRL) framework which uses uncertainty quantification for offline policy learning. In particular, we have three contributions: 1) the method can learn safe and optimal policies through hypothesis testing, 2) ESRL allows for different levels of risk averse implementations tailored to the application context, and finally, 3) we propose a way to interpret ESRL's policy at every state through posterior distributions, and use this framework to compute off-policy value function posteriors. We provide theoretical guarantees for our estimators and regret bounds consistent with Posterior Sampling for RL (PSRL). Sample efficiency of ESRL is independent of the chosen risk aversion threshold and quality of the behavior policy.
3D, virtual, augmented, extended, mixed reality, and extended content forms: The technology and the challenges

3D representations are a new form of information that when coupled with new technology tools, like VR, AR, MR, 3D scanning/printing, and more, offer new support and opportunities for research and pedagogy, often with dramatic leaps in capabilities and results. When combined with a pandemic, the results can be even more dramatic and valued in a variety of collaborative and higher education applications. However, as with any new technology and tools, there also can be sizable challenges that result and will need to be addressed. These include accessibility, 3D object creation, hardware capabilities, storage, and organizing tools. All represent areas where the community of users and their standards organizations, like NISO, need to move aggressively to develop best practices, guidelines, and standards to ensure these new forms of data and technology tools are widely-accessible. This paper provides a high-level overview to introduce people to the new information form, associated technologies, and their challenges.