Reviewerly/exHarmony huggingface dataset:
Syntactic complexity in translated and non-translated texts: A corpus-based study of simplification

This study approaches the investigation of the simplification hypotheses in corpus-based translation studies from a syntactic complexity perspective. The research is based on two comparable corpora, the English monolingual part of COCE (Corpus of Chinese-English) and the native English corpus of FLOB (Freiburg-LOB Corpus of British English). Using the 13 syntactic complexity measures falling into five subconstructs (i.e. length of production unit, amount of subordination, amount of coordination, phrasal complexity and overall sentence complexity), our results show that translation as a whole is less complex compared to non-translation, reflected most prominently in the amount of subordination and overall sentence complexity. Further pairwise comparison of the four subgenres of the corpora shows mixed results. Specifically, the translated news is homogenous to native news as evidenced by the complexity measures; the translated genres of general prose and academic writing are less complex compared to their native counterparts while translated fiction is more complex than non-translated fiction. It was found that mean sentence length always produced a significant effect on syntactic complexity, with higher syntactic complexity for longer sentence lengths in both corpora. ANOVA test shows a highly significant main effect of translation status, with higher syntactic complexity in the non-translated texts (FLOB) than the translated texts (COCE), which provides support for the simplification hypothesis in translation. It is also found that, apart from translation status, genre is an important variable in affecting the complexity level of translated texts. Our study offers new insights into the investigation of simplification hypothesis from the perspective of translation from English into Chinese.
Linearly Replaceable Filters for Deep Network Channel Pruning

Convolutional neural networks (CNNs) have achieved remarkable results; however, despite the development of deep learning, practical user applications are fairly limited because heavy networks can be used solely with the latest hardware and software supports. Therefore, network pruning is gaining attention for general applications in various fields. This paper proposes a novel channel pruning method, Linearly Replaceable Filter (LRF), which suggests that a filter that can be approximated by the linear combination of other filters is replaceable. Moreover, an additional method called Weights Compensation is proposed to support the LRF method. This is a technique that effectively reduces the output difference caused by removing filters via direct weight modification. Through various experiments, we have confirmed that our method achieves state-of-the-art performance in several benchmarks. In particular, on ImageNet, LRF-60 reduces approximately 56% of FLOPs on ResNet-50 without top-5 accuracy drop. Further, through extensive analyses, we proved the effectiveness of our approaches.
TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling

We present a novel approach to the problem of text style transfer.Unlike previous approaches requiring style-labeled training data, our method makes use of readily-available unlabeled text by relying on the implicit connection in style between adjacent sentences, and uses labeled data only at inference time.We adapt T5 (Raffel et al., 2020), a strong pretrained text-to-text model, to extract a style vector from text and use it to condition the decoder to perform style transfer.As our labelfree training results in a style vector space encoding many facets of style, we recast transfers as "targeted restyling" vector operations that adjust specific attributes of the input while preserving others.We demonstrate that training on unlabeled Amazon reviews data results in a model that is competitive on sentiment transfer, even compared to models trained fully on labeled data.Furthermore, applying our novel method to a diverse corpus of unlabeled web text results in a single model capable of transferring along multiple dimensions of style (dialect, emotiveness, formality, politeness, sentiment) despite no additional training and using only a handful of exemplars at inference time.
Classical Arabic Named Entity Recognition Using Variant Deep Neural Network Architectures and BERT

Recurrent Neural Networks (RNNs) and transformers are deep learning models that have achieved remarkable success in several Natural Language Processing (NLP) tasks since they do not rely on handcrafted features nor enormous knowledge resources. Named Entity Recognition (NER) is an essential NLP task that is used in many applications such as information retrieval, question answering, and machine translation. NER aims to locate, extract, and classify named entities into predefined categories such as person, organization and location. Arabic NER is considered a challenging task because of the complexity and the unique characteristics of Arabic. Most of the previous research on deep learning based-Arabic NER focused on Modern Standard Arabic and Dialectal Arabic, which are different variations from Classical Arabic. In this paper, we investigate deep learning-based Classical Arabic NER using different deep neural network architectures and a BERT based contextual language model that is trained on general domain Arabic text. We propose two RNN-based models by fine-tunning the pretrained BERT language model to recognize and classify named entities from Classical Arabic. The pre-trained BERT contextual language model representations were used as input features to a BGRU/BLSTM model and were fine-tuned using a Classical Arabic NER dataset. In addition, we explore variant architectures of the proposed BERT-BGRU/BLSTM-CRF models. Experimentations showed that the BERT-BGRU-CRF model outperformed the other models by achieving an F-measure of 94.76% on the CANERCorpus. To the best of our knowledge, this is the first work that aims to recognize named entities in Classical Arabic using deep learning.
EC-GAN: Low-Sample Classification using Semi-Supervised Algorithms and GANs (Student Abstract)

Semi-supervised learning has been gaining attention as it allows for performing image analysis tasks such as classification with limited labeled data. Some popular algorithms using Generative Adversarial Networks (GANs) for semi-supervised classification share a single architecture for classification and discrimination. However, this may require a model to converge to a separate data distribution for each task, which may reduce overall performance. While progress in semi-supervised learning has been made, less addressed are small-scale, fully-supervised tasks where even unlabeled data is unavailable and unattainable. We therefore, propose a novel GAN model namely External Classifier GAN (EC-GAN), that utilizes GANs and semi-supervised algorithms to improve classification in fully-supervised regimes. Our method leverages a GAN to generate artificial data used to supplement supervised classification. More specifically, we attach an external classifier, hence the name EC-GAN, to the GAN’s generator, as opposed to sharing an architecture with the discriminator. Our experiments demonstrate that EC-GAN's performance is comparable to the shared architecture method, far superior to the standard data augmentation and regularization-based approach, and effective on a small, realistic dataset.
LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer

Many types of text style transfer can be achieved with only small, precise edits (e.g.sentiment transfer from I had a terrible time... to I had a great time...).We propose a coarse-to-fine editor for style transfer that transforms text using Levenshtein edit operations (e.g.insert, replace, delete).Unlike prior single-span edit methods, our method concurrently edits multiple spans in the source text.To train without parallel style text pairs (e.g.pairs of +/-sentiment statements), we propose an unsupervised data synthesis procedure.We first convert text to style-agnostic templates using style classifier attention (e.g.I had a SLOT time...), then fill in slots in these templates using fine-tuned pretrained language models.Our method outperforms existing generation and editing style transfer methods on sentiment (YELP, AMAZON) and politeness (POLITE) transfer.In particular, multi-span editing achieves higher performance and more diverse output than single-span editing.Moreover, compared to previous methods on unsupervised data synthesis, our method results in higher quality parallel style pairs and improves model performance.
Selective Knowledge Distillation for Neural Machine Translation

Neural Machine Translation (NMT) models achieve state-of-the-art performance on many translation benchmarks.As an active research field in NMT, knowledge distillation is widely applied to enhance the model's performance by transferring teacher model's knowledge on each training sample.However, previous work rarely discusses the different impacts and connections among these samples, which serve as the medium for transferring teacher knowledge.In this paper, we design a novel protocol that can effectively analyze the different impacts of samples by comparing various samples' partitions.Based on above protocol, we conduct extensive experiments and find that the teacher's knowledge is not the more, the better.Knowledge over specific samples may even hurt the whole performance of knowledge distillation.Finally, to address these issues, we propose two simple yet effective strategies, i.e., batch-level and global-level selections, to pick suitable samples for distillation.We evaluate our approaches on two large-scale machine translation tasks, WMT'14 English-German and WMT'19 Chinese-English.Experimental results show that our approaches yield up to +1.28 and +0.89 BLEU points improvements over the Transformer baseline, respectively.
PROBLEMATIKA PEMBELAJARAN DARING DI MASA PANDEMI COVID-19

The purpose of this research is to find out the problems of online learning during the Covid-19 pandemic. This research method uses literature study or library research by taking reading sources from secondary data collected through textbooks, scientific journals, e-books and other sources relevant to the research problem. This type of research was analyzed qualitatively with an interactive model consisting of data collection, data reduction, drawing conclusions and verification. The results of this study indicate that online learning is one solution so that the learning process continues during the Covid-19 pandemic. But in this learning there are various problems experienced by various parties, namely educational institutions, educators (teachers and lecturers), students and parents of students. The problems that exist in educational institutions are the lack of availability of telecommunication technology infrastructure, multimedia, information and platforms that support the process of teaching and learning activities online, from educators in the form of limitations in the use of IT, as well as the difficulty of forming student personality characteristics and applying learning media so that students can understand the material presented, from students, namely most students are not familiar with online learning and due to limited internet facilities, from parents the lack of time available to accompany their children during online learning because not all parents can divide their time between work and mentoring children in House.
Super Tickets in Pre-Trained Language Models: From Model Compression to Improving Generalization

The Lottery Ticket Hypothesis suggests that an over-parametrized network consists of "lottery tickets", and training a certain collection of them (i.e., a subnetwork) can match the performance of the full model.In this paper, we study such a collection of tickets, which is referred to as "winning tickets", in extremely over-parametrized models, e.g., pre-trained language models.We observe that at certain compression ratios, the generalization performance of the winning tickets can not only match but also exceed that of the full model.In particular, we observe a phase transition phenomenon: As the compression ratio increases, generalization performance of the winning tickets first improves then deteriorates after a certain threshold.We refer to the tickets on the threshold as "super tickets".We further show that the phase transition is task and model dependent -as the model size becomes larger and the training data set becomes smaller, the transition becomes more pronounced.Our experiments on the GLUE benchmark show that the super tickets improve single task fine-tuning by 0.9 points on BERT-base and 1.0 points on BERT-large, in terms of task-average score.We also demonstrate that adaptively sharing the super tickets across tasks benefits multi-task learning 1 .
Scalable Verification of Quantized Neural Networks

Formal verification of neural networks is an active topic of research, and recent advances have significantly increased the size of the networks that verification tools can handle. However, most methods are designed for verification of an idealized model of the actual network which works over real arithmetic and ignores rounding imprecisions. This idealization is in stark contrast to network quantization, which is a technique that trades numerical precision for computational efficiency and is, therefore, often applied in practice. Neglecting rounding errors of such low-bit quantized neural networks has been shown to lead to wrong conclusions about the network's correctness. Thus, the desired approach for verifying quantized neural networks would be one that takes these rounding errors into account. In this paper, we show that verifying the bit-exact implementation of quantized neural networks with bit-vector specifications is PSPACE-hard, even though verifying idealized real-valued networks and satisfiability of bit-vector specifications alone are each in NP. Furthermore, we explore several practical heuristics toward closing the complexity gap between idealized and bit-exact verification. In particular, we propose three techniques for making SMT-based verification of quantized neural networks more scalable. Our experiments demonstrate that our proposed methods allow a speedup of up to three orders of magnitude over existing approaches.
Proxy Synthesis: Learning with Synthetic Classes for Deep Metric Learning

One of the main purposes of deep metric learning is to construct an embedding space that has well-generalized embeddings on both seen (training) classes and unseen (test) classes. Most existing works have tried to achieve this using different types of metric objectives and hard sample mining strategies with given training data. However, learning with only the training data can be overfitted to the seen classes, leading to the lack of generalization capability on unseen classes. To address this problem, we propose a simple regularizer called Proxy Synthesis that exploits synthetic classes for stronger generalization in deep metric learning. The proposed method generates synthetic embeddings and proxies that work as synthetic classes, and they mimic unseen classes when computing proxy-based losses. Proxy Synthesis derives an embedding space considering class relations and smooth decision boundaries for robustness on unseen classes. Our method is applicable to any proxy-based losses, including softmax and its variants. Extensive experiments on four famous benchmarks in image retrieval tasks demonstrate that Proxy Synthesis significantly boosts the performance of proxy-based losses and achieves state-of-the-art performance. Our implementation is available at github.com/navervision/proxy-synthesis.
Learning from nature for healthcare, energy, and environment

Have you ever looked into the sky and wondered why birds could fly? Or perhaps why the leaves of cacti are needle-like? Throughout evolution, organisms have experimented with various forms and functions for over 3 billion years before the first human manipulations of stone, bone, and antler. From animal teeth to spears, fish fins to paddles, human beings have studied nature not only to satisfy curiosity but, more importantly, to learn from it for innovations and inventions. Along with the development of technology, our understanding of the creatures in nature goes beyond the pure observation of morphology to investigate exquisite composition and knowledge of fundamentals of their unique functions.
Leveraging Type Descriptions for Zero-shot Named Entity Recognition and Classification

A common issue in real-world applications of named entity recognition and classification (NERC) is the absence of annotated data for target entity classes during training.Zeroshot learning approaches address this issue by learning models that can transfer information from observed classes in the training data to unseen classes.This paper presents the first approach for zero-shot NERC, introducing a novel architecture that leverage the fact that textual descriptions for many entity classes occur naturally.Our architecture addresses the zero-shot NERC specific challenge that the not-an-entity class is not well defined, since different entity classes are considered in training and testing.For evaluation, we adapt two datasets, OntoNotes and MedMentions, emulating the difficulty of real-world zero-shot learning by testing models on the rarest entity classes.Our proposed approach outperforms baselines adapted from machine reading comprehension and zero-shot text classification.Furthermore, we assess the effect of different class descriptions for this task.
Focus Attention: Promoting Faithfulness and Diversity in Summarization

Professional summaries are written with document-level information, such as the theme of the document, in mind.This is in contrast with most seq2seq decoders which simultaneously learn to focus on salient content, while deciding what to generate, at each decoding step.With the motivation to narrow this gap, we introduce Focus Attention Mechanism, a simple yet effective method to encourage decoders to proactively generate tokens that are similar or topical to the input document.Further, we propose a Focus Sampling method to enable generation of diverse summaries, an area currently understudied in summarization.When evaluated on the BBC extreme summarization task, two state-of-the-art models augmented with Focus Attention generate summaries that are closer to the target and more faithful to their input documents, outperforming their vanilla counterparts on ROUGE and multiple faithfulness measures.We also empirically demonstrate that Focus Sampling is more effective in generating diverse and faithful summaries than top-k or nucleus samplingbased decoding methods.
AdvPicker: Effectively Leveraging Unlabeled Data via Adversarial Discriminator for Cross-Lingual NER

Neural methods have been shown to achieve high performance in Named Entity Recognition (NER), but rely on costly high-quality labeled data for training, which is not always available across languages.While previous works have shown that unlabeled data in a target language can be used to improve crosslingual model performance, we propose a novel adversarial approach (AdvPicker) to better leverage such data and further improve results.We design an adversarial learning framework in which an encoder learns entity domain knowledge from labeled source-language data and better shared features are captured via adversarial training -where a discriminator selects less language-dependent target-language data via similarity to the source language.Experimental results on standard benchmark datasets well demonstrate that the proposed method benefits strongly from this data selection process and outperforms existing state-ofthe-art methods; without requiring any additional external resources (e.g., gazetteers or via machine translation).
Training Binary Neural Network without Batch Normalization for Image Super-Resolution

Recently, binary neural network (BNN) based super-resolution (SR) methods have enjoyed initial success in the SR field. However, there is a noticeable performance gap between the binarized model and the full-precision one. Furthermore, the batch normalization (BN) in binary SR networks introduces floating-point calculations, which is unfriendly to low-precision hardwares. Therefore, there is still room for improvement in terms of model performance and efficiency. Focusing on this issue, in this paper, we first explore a novel binary training mechanism based on the feature distribution, allowing us to replace all BN layers with a simple training method. Then, we construct a strong baseline by combining the highlights of recent binarization methods, which already surpasses the state-of-the-arts. Next, to train highly accurate binarized SR model, we also develop a lightweight network architecture and a multi-stage knowledge distillation strategy to enhance the model representation ability. Extensive experiments demonstrate that the proposed method not only presents advantages of lower computation as compared to conventional floating-point networks but outperforms the state-of-the-art binary methods on the standard SR networks.
Management Information Systems on Integrated Student and Lecturer Data

One of the most important things in universities is the development of information systems and management of integrated data, important because of the high needs and completeness of attributes in each component related to academic affairs in all universities. Primary data in academics are students and lecturers. Data management and governance are well taken care of in order to deliver information to be accurate, transparency, accountability and actual. Lecturers and students can apply the SDLC model to produce a system that will later improve or replace the old system, through the design and implementation of data management information systems. In the application of policies towards lecturers and students, in this information system will provide formulation and overview to the leadership.
Deep Learning Approach for Negation Handling in Sentiment Analysis

Negation handling is an important sub-task in Sentiment Analysis.Negation plays a significant role in written text.Negation terms in sentence often changes the polarity of entire sentence from positive to negative or vice versa, resulting in the opposite meaning of the sentence than what is observed by the machine learning based linguistic model.As automatic opinion mining has become very important in this digital era, proper handling of negation term is the need of the hour.In any natural language negations can be formulated both explicitly or implicitly while their use is very much domain-specific.Existing negation handling techniques follow rule-based approach and mainly used in medical domain.Due to the complex syntactic structure of negation, it is hard to build general purpose machine learning based negation handling model on user review or conversational text data.In this paper, we investigate negation components i.e., cue and scope in a sentence which determine the polarity shift in sentence.We propose LSTM based deep neural network model for negation handling task where the model automatically learns the negation features from labeled input training dataset.We used ConanDoyle story corpus for model training and testing, which is pre-annotated with negation information.The proposed model first identify negation cues in each sentence and then using bidirectional LSTM extracts the relationship between cue and other words to identify scope of the cue in sentences.We derived word level features for model training to determine correct polarity of the sentence.Result shows that the LSTM based nonlinear language models perform comparatively better than the traditional state of the art SVM, HMM or CRF based models.BiLSTM achieved best result, F1 measures 93.34%, outperform traditional rule based model in negation handling task.