Reviewerly/exHarmony huggingface dataset:
Representing Overlaps in Sequence Labeling Tasks with a Novel Tagging Scheme: Bigappy-Unicrossy

Multiword expression (MWE) identification can be handled by using sequence tagging approach accompanied with stochastic models and variants of IOB tagging scheme. In this paper, we introduce a new tagging scheme called bigappy-unicrossy to rise to the challenge of overlapping MWEs. The bigappy-unicrossy tagging scheme is compared with the two other well-known tagging schemes which are IOB2 and gappy 1-level in the verbal multiword expression (VMWE) identification task using bidirectional Long Short-Term Memory model with a Conditional Random Field layer on top (bidirectional LSTM-CRF). Both the bigappy-unicrossy and the gappy 1-level tagging schemes outperform the IOB2 tagging scheme. The bigappy-unicrossy tagging scheme competes with the gappy 1-level tagging scheme. We believe that our tagging scheme will show better performance on corpora with higher frequency of overlapping cases.
Sentence Compression as a Supervised Learning with a Rich Feature Space

We present a novel supervised approach to sentence compression, based on classification and removal of word sequences generated from subtrees of the original sentence dependency tree. Our system may use any known classifier like Support Vector Machines or Logistic Model Tree to identify word sequences that can be removed without compromising the grammatical correctness of the compressed sentence. We trained our system using several classifiers on a small annotated dataset of 100 sentences, which included around 1500 manually labeled subtrees (removal candidates) represented by 25 features. The highest cross-validation classification accuracy of 80% was obtained with the SMO (Normalized Poly Kernel) algorithm. We evaluated the readability and the informativeness of the sentences compressed by the SMO-based classification model with the help of human raters using a separate benchmark dataset of 200 sentences.
Rethinking Online Knowledge Distillation with Multi-exits

Online knowledge distillation is a method to train multiple networks simultaneously by distilling the knowledge among each other from scratch. An efficient way for this is to attach auxiliary classifiers (called exits) to the main network. However, in this multi-exit approach, there are important questions that have not been answered in previous studies: What structure should be used for exits? What can be a good teacher for distillation? How should the overall training loss be constructed? In this paper, we propose a new online knowledge distillation method using multi-exits by answering these questions. First, we examine the influence of the structure of the exits on the performance of the main network, and propose a bottleneck structure that leads to improved performance for a wide range of main network structures. Second, we propose a new distillation teacher using an ensemble of all the classifiers (main network and exits) by exploiting the diversity in the outputs and features of the classifiers. Third, we propose a new technique to form the overall training loss, which balances classification losses and distillation losses for effective training of the whole network. Our proposed method is termed balanced exit-ensemble distillation (BEED). Experimental results demonstrate that our method achieves significant improvement of classification performance on various popular convolutional neural network (CNN) structures. Code is available at https://github.com/hjdw2/BEED .
Recognizing Weak Signals in News Corpora

This paper presents a suit of experiments carried out with different machine learning systems developed to learn how to find weak signals in news corpora. The task is particularly challenging as the weak signals are not marked at word or sentence level, but rather at document/paragraph level and while there is no explicit definition that strictly applies to them, people are very good at recognizing them. Purposely lacking strict annotation guidelines, a large news corpus was annotated via tacit knowledge and then a supervised learning technique to reproduce the weak signal label was used. A large improvement in weak signals recognition using deep learning over other approaches is reported, useful for investors, entrepreneurs, economists, and normal users, to give them a clue on how to invest.
An Abstractive Text Summarization Using Recurrent Neural Network

With the accelerated advancement of technology and massive content surging over the Internet, it has become an arduous task to abstract the information efficiently. However, automatic text summarization provides an acceptable means for fast procurement of such information in the form of a summary through compression and refinement. Abstractive text summarization, in particular, builds an internal semantic representation of the text and uses natural language generation techniques to create summaries closer to human-generated summaries. This paper uses Long Short Term Memory (LSTM) based Recurrent Neural Network to generate comprehensive abstractive summaries. To train LSTM based model requires a corpus having a significant number of instances containing parallel running article and summary pairs. For this purpose, we have used various news corpus, namely DUC 2003, DUC 2004 and Gigaword corpus, after eliminating the noise and other irrelevant data. Experiments and analyses of this work are performed on a subset of these whole corpora and evaluated using ROUGE evaluation. The experimental result verifies the accuracy and validity of the proposed system.
dpUGC: Learn Differentially Private Representation for User Generated Contents (Best Paper Award, Third Place, Shared)

This paper firstly proposes a simple yet efficient generalized approach to apply differential privacy to text representation (i.e., word embedding). Based on it, we propose a user-level approach to learn personalized differentially private word embedding model on user generated contents (UGC). To our best knowledge, this is the first work of learning user-level differentially private word embedding model from text for sharing. The proposed approaches protect the privacy of the individual from re-identification, especially provide better trade-off of privacy and data utility on UGC data for sharing. The experimental results show that the trained embedding models are applicable for the classic text analysis tasks (e.g., regression). Moreover, the proposed approaches of learning differentially private embedding models are both framework- and data-independent, which facilitates the deployment and sharing. The source code will be available when the paper is published.
Parsing Arabic with a Semi-automatically Generated TAG: Dealing with Linguistic Phenomena

Arabic is a challenging language when it comes to grammar production and parsing. It combines complex linguistic phenomena with a rich morphology that make its processing particularly ambiguous. This leaded us to choose the Tree-Adjoining Grammar (TAG) formalism. Indeed, TAG provides sufficient constraints for handling diverse linguistic phenomena and seems to be adequate to represent Arabic syntactic structures. In this paper, we present a semi-automatically generated TAG for modern standard Arabic using a compiler and a metagrammatical description language called XMG (eXtensible MetaGrammar). We describe the linguistic coverage of our grammar, and show how we used TAG and XMG’s properties to define in an expressive and concise way different linguistic phenomena. To check the coverage of our grammar, we have set up a development environment including a parser and using a test corpus of linguistic phenomena gathering both grammatical and ungrammatical sentences.
AFF-CAM: Adaptive Frequency Filtering Based Channel Attention Module

Locality from bounded receptive fields is one of the biggest problems that needs to be solved in convolutional neural networks. Meanwhile, operating convolutions in frequency domain provides complementary viewpoint to this dilemma, as a point-wise update in frequency domain can globally modulate all input features involved in Discrete Cosine Transform. However, Discrete Cosine Transform concentrates majority of its information in a handful of coefficients in lower regions of frequency spectrum, often discarding other potentially useful frequency components, such as those of middle and high frequency spectrum. We believe valuable feature representations can be learned not only from lower frequency components, but also from such disregarded frequency distributions. In this paper, we propose a novel Adaptive Frequency Filtering based Channel Attention Module (AFF-CAM), which exploits non-local characteristics of frequency domain and also adaptively learns the importance of different bands of frequency spectrum by modeling global cross-channel interactions, where each channel serves as a distinct frequency distribution. As a result, AFF-CAM is able to re-calibrate channel-wise feature responses and guide feature representations from spatial domain to reason over high-level, global context, which simply cannot be obtained from local kernels in spatial convolutions. Extensive experiments are conducted on ImageNet-1K classification and MS COCO detection benchmarks to validate our AFF-CAM. By effectively aggregating global information of various frequency spectrum from frequency domain with local information from spatial domain, our method achieves state-of-the-art results compared to other attention mechanisms.
SMGKM: An Efficient Incremental Algorithm for Clustering Document Collections

Given a large unlabeled document collection, the aim of this paper is to develop an accurate and efficient algorithm for solving the clustering problem over this collection. Document collections typically contain tens or hundreds of thousands of documents, with thousands or tens of thousands of features (i.e., distinct words). Most existing clustering algorithms struggle to find accurate solutions on such large data sets. The proposed algorithm overcomes this difficulty by an incremental approach, incrementing the number of clusters progressively from an initial value of one to a set value. At each iteration, the new candidate cluster is initialized using a partitioning approach which is guaranteed to minimize the objective function. Experiments have been carried out over six, diverse datasets and with different evaluation criteria, showing that the proposed algorithm has outperformed comparable state-of-the-art clustering algorithms in all cases.
Generating Jigsaw Puzzles and an AI Powered Solver

This paper tackles the problem of assembling a jigsaw puzzle, starting only from a picture of the scrambled jigsaw puzzle pieces on a random, textured background. This manuscript discusses previous approaches in dealing with the jigsaw puzzle problem and brings two contributions: an open source tool for creating realistic scrambled jigsaw puzzles meant to serve as a foundation for further research in the field; and an end to end AI based solution taking advantage of the convolutional neural network architecture, capable of solving a scrambled jigsaw puzzle of unknown pictorial and with an unknown, uniformly textured, background. The lessons and techniques learned in engaging with the jigsaw puzzle problem can be further used in approaching the more general and complex problem of Protein-Protein interaction prediction.
Filter Pruning via Automatic Pruning Rate Search

Model pruning is important for deploying models on devices with limited resources. However, the searching of optimal pruned model is still a significant challenge due to the large space to be exploited. In this paper, we propose an Automatic Pruning Rate Search(APRS) method to achieve automatic pruning. We reveal the connection between the model performance and Wasserstein distance to automatic searching optimal pruning rate. To reduce the search space, we quantify the sensitivity of each filter layer by layer and reveal the connection between model performance and Wasserstein distance. We introduce an end-to-end optimization method called Pareto plane to automatically search for the pruning rate to fit the overall size of the model. APRS can obtain more compact and efficient pruning models. To verify the effectiveness of our method, we conduct extensive experiments on ResNet, VGG and DenseNet, and the results show that our method outperforms the state-of-the-art methods under different parameter settings.
Multi-lingual Event Identification in Disaster Domain

Information extraction in disaster domain is a critical task for effective disaster management. A high quality event detection system is the very first step towards this. Since disaster annotated data-sets are not available in Indian languages, we first create and annotate a dataset in three different languages, namely Hindi, Bengali and English. The data was crawled from the different news websites and annotated with expert annotators using a proper annotation guidelines. The events in the dataset belong to 35 different disaster classes. We then build a deep ensemble architecture based on Convolution Neural Network (CNN) and Bi-directional Long Short Term Memory (Bi-LSTM) network as the base learning models. This model is used to identify event words and phrases along with its class from the input sentence. Since our data is sparse, the model yields a very low F1-score in all the three languages. To mitigate the data sparsity problem we make use of multi-lingual word embedding so that joint training of all the languages could be done. To accommodate joint training we modify our model to contain language-specific layers so that the syntactic differences between the languages can be taken care of by these layers. By using multi-lingual embedding and training the whole dataset on our proposed model, the performance of event detection in each language improves significantly. We also report further analysis of language-wise and class-wise improvements of each language and event classes.
A Decision-Level Approach to Multimodal Sentiment Analysis

There has been near exponential increase in the use of images and video on various Social Media platforms in the last few years, in place of or in addition to the use of plain text. Automated sentiment analysis, at its core, is the capturing of human emotion by machine - the addition of image and video to social media output had made this already challenging task even greater. In this paper, we propose a multimodal, decision-level based approach to sentiment analysis (SA) of Twitter feeds. The solution proposed and outlined in this paper, combines the sentiment analysis scoring of not just text-based output but integrates SA scoring generated from analysis of image captions. For our experiments, we focused on politics and on two political topics (Trump/Brexit) that are generating a lot of discussion and debate on Twitter. We chose the political domain given the power that Social Media has on possibly influencing voters ( https://www.theguardian.com/technology/2016/jul/31/trash-talk-how-twitter-is-shaping-the-new-politics ) and the ‘strong’ opinions that are expressed in this area.
Take Help from Elder Brother: Old to Modern English NMT with Phrase Pair Feedback

Due to the ever-changing nature of the human language and the variations in writing style, age-old texts in one language may be incomprehensible to a modern reader. In order to make these texts familiar to the modern reader, we need to rewrite them manually. But this is not always feasible if the volume of texts is very large. In this paper, we present this rewriting task as a neural machine translation (NMT) problem. We propose an effective approach for training NMT system using a tiny parallel corpus comprising of only 2.7 k parallel sentences. We inject parallel phrase pairs extracted using Statistical Machine Translation (SMT) as additional training examples to NMT. We choose publicly available old-modern English parallel texts for our experiments. Evaluation results show that our proposed approach outperforms the baseline NMT system by more than 18 BLEU points without using any additional training data.
PEDTrans: A Fine-Grained Visual Classification Model for Self-attention Patch Enhancement and Dropout

Fine-grained visual classification (FGVC) is an essential and challenging classification task in computer visual classification, aiming to identify different cars and birds. Recently, most studies use a convolutional neural network combined with an attention mechanism to find discriminant regions to improve algorithm accuracy automatically. However, the discriminant regions selected by the convolutional neural network are extensive. Vision Transformer divides the image into patches and relies on self-attention to select more accurate discriminant regions. However, the Vision Transformer model ignores the response between local patches before patch embedding. In addition, patches usually have high similarity, and they are considered redundant. Therefore, we propose a PEDTrans model based on Vision Transformer. The model has a patch enhancement module based on attention mechanism and a random similar group patch discarding module based on similarity. These two modules can establish patch local feature relationships and select patches that are easier to distinguish between images. Combining these two modules with the Vision Transformer backbone network can improve the fine-grained visual classification accuracy. We employ commonly used fine-grained visual classification datasets CUB-200-2011, Stanford Cars, Stanford Dogs and NABirds to get advanced results.
Meta-prototype Decoupled Training for Long-Tailed Learning

Long-tailed learning aims to tackle the crucial challenge that head classes dominate the training procedure under severe class imbalance in real-world scenarios. Supervised contrastive learning has turned out to be worth exploring research direction, which seeks to learn class-specific feature prototypes to enhance long-tailed learning performance. However, little attention has been paid to how to calibrate the empirical prototypes which are severely biased due to the scarce data in tail classes. Without the aid of correct prototypes, these explorations have not shown the significant promise expected. Motivated by this, we propose the meta-prototype contrastive learning to automatically learn the reliable representativeness of prototypes and more discriminative feature space via a meta-learning manner. In addition, on top of the calibrated prototypes, we leverage it to replace the mean of class statistics and predict the targeted distribution of balanced training data. By this procedure, we formulate the feature augmentation algorithm which samples additional features from the predicted distribution and further balances the over-whelming dominance severity of head classes. We summarize the above two stages as the meta-prototype decouple training scheme and conduct a series of experiments to validate the effectiveness of the framework. Our method outperforms previous work with a large margin and achieves state-of-the-art performance on long-tailed image classification and semantic segmentation tasks (e.g., we achieve 55.1% overall accuracy with ResNetXt-50 in ImageNet-LT).
Features of the Motivational and Value Sphere of Students of Different Courses of Study

The purpose of the study is to determine the features of the motivational and value sphere of students of different courses of study. It was suggested that: 1. there are specific features of the motivational-value sphere of students studying at different courses; 2. there is a relationship between the course of study and motivational-value indicators of students. Research methods: "Study of motives of students' educational activity" modified by A.A. Rean, V.A. Yakunin, "Morphological test of life values" by V. F. Sopov, L. V. Karpushina, "Anchors of career" adaptation by V.A. Chiker, V.E. Vinokurova. The study was attended by students of 1-4 courses of the FSUE VO "ZabGU", in the direction of training 44.03.02 Psychological and pedagogical education. 75 people participated in the study. As a result of the study, it was found that the majority of students, regardless of the course of study, have formed spiritual, moral and altruistic values of their future profession. The educational motivation of first and second year students is more focused on the successful implementation of educational and cognitive activities. External motivations of educational motivation, such as social approval/ punishment for senior students are non-priority and ineffective in the process of educational activity. The career orientations of junior students are of a mixed nature. For senior students, the nature of career orientations becomes more differentiated, a limited number of orientations dominate. A special contribution of the authors to the study of the topic is the established relationship between the course of study and indicators of the motivational and value sphere, such as: to ensure the success of future professional activity, one's own prestige, high financial status and the preservation of one's own individuality, as well as the establishment of an inverse relationship between the course of study and the indicator of successfully studying, passing exams with excellent and good grades.
Automatic Creation of a Pharmaceutical Corpus Based on Open-Data

A large amount of information related to the pharmaceutical industry is published through the web, but there are few tools that allow for automatic analysis. The pharmaceutical area at international level, especially for English language, has a large number of virtual lexical tools, that support not only the research but the software development, while for Spanish language, are few lexical tools and less for the particularities of a country as Colombia. This paper presents the pharmaceutical corpus generation based on open data of Colombian medicines published monthly by the National Institute of Medicines and Food Surveillance (INVIMA). A model has been developed that combines the concepts of corpus and ontology, and the model is structured through a multi-related graph. This model is implemented in a graph-oriented database, because it has been shown to manage this type of structures and since they are based on a mathematical theory, graph-oriented database allows to find patterns and relationships that would otherwise not be possible. For the creation of the corpus, a Crawler was developed to download and control the documents, and through text processing and the proper algorithm are stored in the graph-oriented database (Neo4j).
Learning Common and Specific Visual Prompts for Domain Generalization

Although fine-tuning a pre-trained large-scale model has become an effective method for domain generalization, domain shifts still issue a huge challenge for successfully transferring models to unseen test domains. In this paper, we study how to effectively adapt pre-trained vision Transformers for domain generalization problems in image classification. To this end, this paper proposes a novel Common-Specific Visual Prompt Tuning (CSVPT) method to transfer large-scale vision Transformer models to unknown test domains. Different from existing methods which learn fixed visual prompts for each task, CSVPT jointly learns domain-common prompts to capture the task context and sample-specific prompts to capture information about data distribution, which are generated for each sample through a trainable prompt-generating module (PGM). Combining the domain-common prompts and the sample-specific prompts, visual prompts learned by CSVPT are conditioned on each input sample rather than fixed once learned, which helps out-of-distribution generalization. Extensive experimental results show the effectiveness of CSVPT, and CSVPT with the backbone ViT-L/14 achieves state-of-the-art (SOTA) performance on five widely used benchmark datasets.