Reviewerly/exHarmony huggingface dataset:
Deep Archimedean Copulas

A central problem in machine learning and statistics is to model joint densities of random variables from data. Copulas are joint cumulative distribution functions with uniform marginal distributions and are used to capture interdependencies in isolation from marginals. Copulas are widely used within statistics, but have not gained traction in the context of modern deep learning. In this paper, we introduce ACNet, a novel differentiable neural network architecture that enforces structural properties and enables one to learn an important class of copulas--Archimedean Copulas. Unlike Generative Adversarial Networks, Variational Autoencoders, or Normalizing Flow methods, which learn either densities or the generative process directly, ACNet learns a generator of the copula, which implicitly defines the cumulative distribution function of a joint distribution. We give a probabilistic interpretation of the network parameters of ACNet and use this to derive a simple but efficient sampling algorithm for the learned copula. Our experiments show that ACNet is able to both approximate common Archimedean Copulas and generate new copulas which may provide better fits to data.
Analytic Characterization of the Hessian in Shallow ReLU Models: A Tale of Symmetry

We consider the optimization problem associated with fitting two-layers ReLU networks with respect to the squared loss, where labels are generated by a target network. We leverage the rich symmetry structure to analytically characterize the Hessian at various families of spurious minima in the natural regime where the number of inputs $d$ and the number of hidden neurons $k$ is finite. In particular, we prove that for $d\ge k$ standard Gaussian inputs: (a) of the $dk$ eigenvalues of the Hessian, $dk - O(d)$ concentrate near zero, (b) $\Omega(d)$ of the eigenvalues grow linearly with $k$. Although this phenomenon of extremely skewed spectrum has been observed many times before, to our knowledge, this is the first time it has been established {rigorously}. Our analytic approach uses techniques, new to the field, from symmetry breaking and representation theory, and carries important implications for our ability to argue about statistical generalization through local curvature.
Introduction to a <i>Special Issue on Human Rights in the Time of COVID-19</i>

The urgent need to create a special issue on the human rights impact of the COVID-19 pandemic became apparent as soon as it began to unfold in early spring 2020. As an editorial team, we are deeply...
Attitudinal factors among elementary school teacher education students

&lt;span&gt;In second or foreign language learning, attitudinal factors were one of the key factors that determine language success and achievement. In Indonesia, foreign language (English) has been taught since elementary school, however research showed most Indonesian language learners in English showed unsatisfactorily result. This research aimed at disclosing the students’ attitudinal factors in learning foreign language by exploring their first experience in learning foreign language. This qualitative study employed exploratory case study design to interview students’ attitudinal factors by exploring the students past experience at first time learning foreign language. These students were from different education unit background when they were in elementary school. This study applied thematic analysis to interpret and analyze the data. The findings showed in case of attitudinal factors were categorized into three components that were cognitive, affective, and behavior. In case of cognitive and affective components, those who had good experiences felt happy in their first experience in foreign language and the vice versa. However, both were having similar learning behavior when they serve as an English teacher, they adopted particular learning behavior that was able to make students enjoy the foreign language learning.&lt;/span&gt;
Summary of Research on Intelligent Education in China in Recent Ten Years-Based on Co-Word Clustering Analysis

In recent years, intelligent education has gradually become a hot topic in the field of educational informatics.With the help of SATI4.0 and SPSS22.0software, this paper makes a visual analysis of the intelligent education literature samples from the perspectives of annual publication trend, major journals and research topics.The purpose is to illustrate the current hot spots and status quo of intelligent education research through data, so as to provide useful reference for the further development of intelligent education research.The results show that the current research on intelligent education is in the initial stage and generally on the rise, with interdisciplinary features.At the same time, there are still some problems in the research, such as the incomplete theoretical system, uneven research development in various fields, and weak research on the application of theory to practice.The future domestic education should be perfect intelligent education theory system, strengthen the practice research, and expand the study further breakthrough in breadth and depth of research.
Data-driven Summarization of Broadcasted Cycling Races by Automatic Team and Rider Recognition

The number of spectators for cycling races broadcasted on television is decreasing each year. More dynamic and personalized reporting formats are needed to keep the viewer interested. In this paper, we propose a methodology for data-driven summarization, which allows end-users to query for personalized stories of a race, tailored to their needs (such as the length of the clip and the riders and/or teams that they are interested in). The automatic summarization uses a combination of skeleton-based rider pose detection and pose-based recognition algorithms of the team jerseys and rider faces/numbers. Evaluation on both cyclocross and road cycling races show that there is certainly potential in this novel methodology.
Special Issue on Unmanned Aerial Vehicles (UAVs)

This editorial paper was a special issue of Applied Sciences belonging to the section of mechanical engineering in MDPI journal and summarized the collected manuscripts regarding the unmanned aerial vehicles (UAVs) related technologies, including communication, control, collision avoidance, modeling, path planning, human-machine interface (HMI), artificial intelligence (AI), etc. Chronologically, this special issue was started to be coordinated at the end of Oct 2018, prepared for a month and opened to collect manuscripts from the middle of Nov 2018 until the end of Dec 2019. During almost a year, 26 papers were published online out of 50 submitted papers which results in 52% acceptance rate.
Efficient SSE With Forward ID-Privacy and Authentication in the Multi-Data-Owner Settings

Based on Sun et al.'s multi-client symmetric searchable encryption (SSE) scheme (at ESORICS 2016), and combining Zhao's identity-concealed authenticated encryption (CCS 2016), a new SSE scheme with multi-data-owner functionalities is proposed. By setting two key generation centers, our scheme first implements multi-data-owner SSE. In particular, compared with Sun et al.'s scheme, the new scheme not only meets the same security requirements stated by them, but also further strengthens the securities of the same category relevant scheme by providing identity-concealment, authentication of data user to server and confidentiality of search token. The identity-concealment aims to provide privacy protection (Forward ID-Privacy) for data users by hiding their identity information, while the authentication is to resist the camouflage attack by applying certificate-based mechanism to our scheme. In particular, the confidentiality of the search token provides replay-attack-resistant by encrypting the plaintext search token generated by data user. While in other works, the adversary can employ the previously generated plaintext search tokens to force the server to perform the same search queries. Furthermore, by efficiency analysis, our scheme reaches almost the same level of efficiency as Sun et al.'s scheme.
Balancing privacy and accountability in digital payment methods using zk-SNARKs

In this paper we propose and implement a digital permissioned decentralized anonymous payment scheme that finds a balance between anonymity and auditability.This approach allows banks to ensure that their clients are not participating in illegal financial transactions, whilst clients stay in control over their sensitive, personal information.Existing anonymous payment schemes often provide good privacy, but only little or mostly no auditability.We provide both by extending the Zerocash zk-SNARK based approach and adding functionality that allows for customer due diligence 'at the gate'.Clients can do fully anonymous transactions up to a certain amount per time unit and larger transactions are forced to include verifiably encrypted transactions details that can only be opened by a select group of 'judges'.
Unsupervised Cross-Lingual Adaptation of Dependency Parsers Using CRF Autoencoders

We consider the task of cross-lingual adaptation of dependency parsers without annotated target corpora and parallel corpora.Previous work either directly applies a discriminative source parser to the target language, ignoring unannotated target corpora, or employs an unsupervised generative parser that can leverage unannotated target data but has weaker representational power than discriminative parsers.In this paper, we propose to utilize unsupervised discriminative parsers based on the CRF autoencoder framework for this task.We train a source parser and use it to initialize and regularize a target parser that is trained on unannotated target data.We conduct experiments that transfer an English parser to 20 target languages.The results show that our method significantly outperforms previous methods. 1
R-BERT FOR RELATIONSHIP EXTRACTION ON RUSSIAN BUSINESS DOCUMENTS

This paper provides results of participation in the Russian Relation Extraction for Business shared task (RuREBus) within DialogueEvaluation 2020. Our team took the first place among 5 other teams in Relation Extraction with Named Entities task. The experiments showed that the best model is based on R-BERT model. R-BERT achieved significant result in comparison with models based on Convolutional or Recurrent Neural Networks on the SemEval-2010 task 8 relational dataset. In order to adapt this model to RuREBus task we also added some modifications like negative sampling. In addition, we have tested other models for Relation Extraction and Named Entity Recognition tasks.
Aplikasi Perhitungan PPH 21 Dan PPH 26 Karyawan Berbasis Web

Income Tax PPh 21/26 is a tax on I ncome in the form of salaries, wages, honoraria, allowances, and other payments in whatever name and form in connection with work or services. There are also many taxpayers who do not understand how to calculate Income Tax 21/26 on the income they get from their work. therefore, the public must further increase self-awareness to pay taxes and report annual tax returns and knowledge of how to calculate income tax is also useful for taxpayers in the tax reporting process. The purpose of this research is to know and analyze the procedure for calculating the PPh 21/26 employee's income tax calculation system and designing and building an application for calculating the PPh 21/26 employee's income tax. With the application for calculating employee income tax article 21/26, it will make it easier for employees to make tax payment transactions. In the PPh 21/26 Calculation Application the system displays tax payment data, employee data and user data. In making it, the author uses the Codeigniter Framework application with the hypertext preprocessor (PHP) programming language. This program also uses several supporting software in making the Web such as Xampp as a local server, Google Chrome as a Web browser and Mysql as a database server
Achieving Equalized Odds by Resampling Sensitive Attributes

We present a flexible framework for learning predictive models that approximately satisfy the equalized odds notion of fairness. This is achieved by introducing a general discrepancy functional that rigorously quantifies violations of this criterion. This differentiable functional is used as a penalty driving the model parameters towards equalized odds. To rigorously evaluate fitted models, we develop a formal hypothesis test to detect whether a prediction rule violates this property, the first such test in the literature. Both the model fitting and hypothesis testing leverage a resampled version of the sensitive attribute obeying equalized odds, by construction. We demonstrate the applicability and validity of the proposed framework both in regression and multi-class classification problems, reporting improved performance over state-of-the-art methods. Lastly, we show how to incorporate techniques for equitable uncertainty quantification---unbiased for each group under study---to communicate the results of the data analysis in exact terms.
Towards Domain-Independent Text Structuring Trainable on Large Discourse Treebanks

Text structuring is a fundamental step in NLG, especially when generating multi-sentential text.With the goal of fostering more general and data-driven approaches to text structuring, we propose the new and domain-independent NLG task of structuring and ordering a (possibly large) set of EDUs.We then present a solution for this task that combines neural dependency tree induction with pointer networks and can be trained on large discourse treebanks that have only recently become available.Further, we propose a new evaluation metric that is arguably more suitable for our new task compared to existing content ordering metrics.Finally, we empirically show that our approach outperforms competitive alternatives on the proposed measure and is equivalent in performance with respect to previously established measures.
Coming to Terms: Automatic Formation of Neologisms in Hebrew

Spoken languages are ever-changing, with new words entering them all the time.However, coming up with new words (neologisms) today relies exclusively on human creativity.In this paper we propose a system to automatically suggest neologisms.We focus on the Hebrew language as a test case due to the unusual regularity of its noun formation.User studies comparing our algorithm to experts and nonexperts demonstrate that our algorithm is capable of generating high-quality outputs, as well as enhance human creativity.More broadly, we seek to inspire more computational work around the topic of linguistic creativity, which we believe offers numerous unexplored opportunities.
Punctuation Restoration using Transformer Models for Resource-Rich and -Poor Languages

Punctuation restoration is a common post-processing problem for Automatic Speech Recognition (ASR) systems. It is important to improve the readability of the transcribed text for the human reader and facilitate NLP tasks. Current state-of-art address this problem using different deep learning models. Recently, transformer models have proven their success in downstream NLP tasks, and these models have been explored very little for the punctuation restoration problem. In this work, we explore different transformer based models and propose an augmentation strategy for this task, focusing on high-resource (English) and low-resource (Bangla) languages. For English, we obtain comparable state-of-the-art results, while for Bangla, it is the first reported work, which can serve as a strong baseline for future work. We have made our developed Bangla dataset publicly available for the research community.
Exploring Span Representations in Neural Coreference Resolution

In coreference resolution, span representations play a key role to predict coreference links accurately.We present a thorough examination of the span representation derived by applying BERT on coreference resolution (Joshi et al., 2019) using a probing model.Our results show that the span representation is able to encode a significant amount of coreference information.In addition, we find that the head-finding attention mechanism involved in creating the spans is crucial in encoding coreference knowledge.Last, our analysis shows that the span representation cannot capture non-local coreference as efficiently as local coreference.
A Survey about Deep Learning for Constellation Design in Communications

The performance of communication systems depends on the choice of constellations, designed in an end-to-end manner. In case of a mathematical intractability, either because of complexity or even lack of channel model only sub-optimal solutions can be provided with an analytical approach. We present end-to-end learning, a recent technique in communications to learn optimal transmitter and receiver architectures based on deep neural networks (DNNs) architectures. We discuss the cases in which this technique has been used to design constellations, where a mathematical analysis is repressed due to the channel model intractability.
The basics of designing a digital simulation for the preparation of teachers of mathematics

Introduction. The article discusses the positive aspects of the introduction of digital simulators of pedagogical activity into the process of professional training of teachers. The subject of the research is the most significant technical factors affecting the design of a digital simulator for professional training of mathematics teachers. Methodological approaches to simulating the process of learning to find solutions to problems are considered. Materials and Methods. In the article there was conducted theoretical analysis of the complex domestic and foreign scientific sources (articles and publications on the research topic, the results of dissertational researches in pedagogics and psychology), which generalize and systematize international and Russian experience in relation to teacher training, and in terms of the characteristics of different virtual simulators. There were applied a method of theoretical generalization, content analysis, technical algorithm for using computer programs in the designing of a simulator. Results. As a result of the authors’ analysis and interpretation of available sources, the data of international and Russian studies are summarized and systematized. Various platforms and technical characteristics that allow implementing simulators are analyzed. The technical data obtained during the analysis allowed us to identify and justify the process of creating the simulator platform. The visual concept of a simulator, which provides for the use of vector and raster graphics packages, is presented. Discussion and Conclusions. The article presents the stages of development and implementation of simulators in the professional training of teachers. The new digital educational environment with the use of computer modeling capabilities will allow future teachers to master professional competencies by training virtual students.
Continual Learning with Node-Importance based Adaptive Group Sparse Regularization

We propose a novel regularization-based continual learning method, dubbed as Adaptive Group Sparsity based Continual Learning (AGS-CL), using two group sparsity-based penalties. Our method selectively employs the two penalties when learning each node based its the importance, which is adaptively updated after learning each new task. By utilizing the proximal gradient descent method for learning, the exact sparsity and freezing of the model is guaranteed, and thus, the learner can explicitly control the model capacity as the learning continues. Furthermore, as a critical detail, we re-initialize the weights associated with unimportant nodes after learning each task in order to prevent the negative transfer that causes the catastrophic forgetting and facilitate efficient learning of new tasks. Throughout the extensive experimental results, we show that our AGS-CL uses much less additional memory space for storing the regularization parameters, and it significantly outperforms several state-of-the-art baselines on representative continual learning benchmarks for both supervised and reinforcement learning tasks.
Fast Packet Inspection for End-To-End Encryption

With the recent development and popularization of various network technologies, communicating with people at any time, and from any location, using high-speed internet, has become easily accessible. At the same time, eavesdropping, data interception, personal data leakage, and distribution of malware during the information transfer process have become easier than ever. Recently, to respond to such threats, end-to-end encryption (E2EE) technology has been widely implemented in commercial network services as a popular information security system. However, with the use of E2EE technology, it is difficult to check whether an encrypted packet is malicious in an information security system. A number of studies have been previously conducted on deep packet inspection (DPI) through trustable information security systems. However, the E2EE is not maintained when conducting a DPI, which requires a long inspection time. Thus, in this study, a fast packet inspection (FPI) and its frame structure for quickly detecting known malware patterns while maintaining E2EE are proposed. Based on the simulation results, the proposed FPI allows for inspecting packets approximately 14.4 and 5.3 times faster, respectively, when the inspection coverage is 20% and 100%, as compared with a DPI method under a simulation environment in which the payload length is set to 640 bytes.
On the Branching Bias of Syntax Extracted from Pre-trained Language Models

Many efforts have been devoted to extracting constituency trees from pre-trained language models, often proceeding in two stages: feature definition and parsing.However, this kind of methods may suffer from the branching bias issue, which will inflate the performances on languages with the same branch it biases to.In this work, we propose quantitatively measuring the branching bias by comparing the performance gap on a language and its reversed language, which is agnostic to both language models and extracting methods.Furthermore, we analyze the impacts of three factors on the branching bias, namely parsing algorithms, feature definitions, and language models.Experiments show that several existing works exhibit branching biases, and some implementations of these three factors can introduce the branching bias.
Joint Intent Detection and Entity Linking on Spatial Domain Queries

Continuous efforts have been devoted to language understanding (LU) for conversational queries with the fast and wide-spread popularity of voice assistants.In this paper, we first study the LU problem in the spatial domain, which is a critical problem for providing location-based services by voice assistants, but is without in-depth investigation in existing studies.Spatial domain queries have several unique properties making them be more challenging for language understanding than common conversational queries, including lexicalsimilar but diverse intents and highly ambiguous words.Thus, a special tailored LU framework for spatial domain queries is necessary.To the end, a dataset SMQ was extracted and annotated based on the real-life queries from a voice assistant service.We then proposed a new multi-task framework MELIP that jointly learns the intent detection and entity linking tasks on the SMQ with invented hierarchical intent detection method and triple-scoring mechanism for entity linking.A specially designed spatial GCN is also utilized to model spatial context information among entities.We have conducted extensive experimental evaluations with state-of-the-art entity linking and intent detection methods, which demonstrated that MELIP can outperform all baselines with a significant margin.