Reviewerly/exHarmony huggingface dataset:
Keyword Extraction Method for Complex Nodes Based on TextRank Algorithm

Keywords are important way for people to quickly understand content of document and grasp subject, and keyword extraction technology is significant way to quickly obtain core meaning of text information, which has wide range of applications in fields such as intelligence, journalism, information retrieval and natural language understanding. However, traditional TextRank algorithm refers to local co-occurrence relationship among text words, which does not pay much attention to complex network structure characteristics of word graphs. Therefore, structure of network is adjusted by removing nodes to separate sub-networks with layers. Moreover, taking into account complex network structure characteristics of word nodes, method of word node removal is introduced as well. Meanwhile, value of sliding window is increased so that ranking can be obtained through multiple iterations, and then one of the highest ranking keyword nodes will be removed in turn. Besides it, Keyword extraction is then performed on each subtopic where sub-keywords are determined based on ranking of candidate keywords, and key nodes of text network are added to sequence so that keyword extraction can be achieved, which achieves improvement of traditional TextRank algorithm, and accuracy, recall, and F value are all improved as well.
Alternative quantitative methods in psycholinguistics: Implications for theory and design

We describe three different methods that are appropriate to analyze various types of psycholinguistic data.We discuss some of the strengths and weaknesses of each and their suitability according to characteristics of the data.Methods include analysis of variance (ANOVA), linear mixed-effects modeling (LME) and generalized additive mixed models (GAMM).
Automatic Truecasing of Video Subtitles Using BERT: A Multilingual Adaptable Approach

This paper describes an approach for automatic capitalization of text without case information, such as spoken transcripts of video subtitles, produced by automatic speech recognition systems. Our approach is based on pre-trained contextualized word embeddings, requires only a small portion of data for training when compared with traditional approaches, and is able to achieve state-of-the-art results. The paper reports experiments both on general written data from the European Parliament, and on video subtitles, revealing that the proposed approach is suitable for performing capitalization, not only in each one of the domains, but also in a cross-domain scenario. We have also created a versatile multilingual model, and the conducted experiments show that good results can be achieved both for monolingual and multilingual data. Finally, we applied domain adaptation by finetuning models, initially trained on general written data, on video subtitles, revealing gains over other approaches not only in performance but also in terms of computational cost.
Is Finance-Growth Nexus Linear in Selected Countries of Middle East and Northern Africa?

The present study re-examines the impact of financial development on economic growth in resource-rich Middle East and North Africa (MENA) countries over 1987-2015. Although several studies investigate the finance-growth nexus, none emphasized the nature of this relationship in MENA. In the long run, an inverted U-shaped association between finance and growth is indicated when using pooled mean group estimations. However, the relationship is not significant in the short run. The outcomes suggest that financial development is significantly and positively affiliated with economic growth up to a certain level. After this turning point, additional financial development tends to adversely affect economic growth. The existence of an inverse U-shape association between financial development and economic growth was confirmed by the estimation of the U-test. The outcomes of our study are important to policymakers, in terms of optimizing the necessary and limit of financial development to ensure maximum benefit for the whole economy through the banking sector.
Fast CNN Stereo Depth Estimation through Embedded GPU Devices

Current CNN-based stereo depth estimation models can barely run under real-time constraints on embedded graphic processing unit (GPU) devices. Moreover, state-of-the-art evaluations usually do not consider model optimization techniques, being that it is unknown what is the current potential on embedded GPU devices. In this work, we evaluate two state-of-the-art models on three different embedded GPU devices, with and without optimization methods, presenting performance results that illustrate the actual capabilities of embedded GPU devices for stereo depth estimation. More importantly, based on our evaluation, we propose the use of a U-Net like architecture for postprocessing the cost-volume, instead of a typical sequence of 3D convolutions, drastically augmenting the runtime speed of current models. In our experiments, we achieve real-time inference speed, in the range of 5-32 ms, for 1216 × 368 input stereo images on the Jetson TX2, Jetson Xavier, and Jetson Nano embedded devices.
Creating Classification Models from Textual Descriptions of Companies Using Crunchbase

This paper compares different models for multilabel text classification, using information collected from Crunchbase, a large database that holds information about more than 600000 companies. Each company is labeled with one or more categories, from a subset of 46 possible categories, and the proposed models predict the categories based solely on the company textual description. A number of natural language processing strategies have been tested for feature extraction, including stemming, lemmatization, and part-of-speech tags. This is a highly unbalanced dataset, where the frequency of each category ranges from 0.7% to 28%. Our findings reveal that the description text of each company contain features that allow to predict its area of activity, expressed by its corresponding categories, with about 70% precision, and 42% recall. In a second set of experiments, a multiclass problem that attempts to find the most probable category, we obtained about 67% accuracy using SVM and Fuzzy Fingerprints. The resulting models may constitute an important asset for automatic classification of texts, not only consisting of company descriptions, but also other texts, such as web pages, text blogs, news pages, etc.
Aspect Term Extraction Using Deep Learning Model with Minimal Feature Engineering

With the explosive growth of social media on the Web, opinion mining has been extensively investigated and consists of the automatic identification and extraction of opinions, emotions, and sentiments from text and multimedia data. One of the tasks involved in opinion mining is Aspect Term Extraction (ATE) which aims at identifying aspects (attributes or characteristics) that have been explicitly evaluated in a sentence or a document. For example, in the sentence "The picture quality of this camera is amazing", the aspect term is "picture quality". This work proposes POS-AttWD-BLSTM-CRF, a neural network architecture using a deep learning model, and minimal feature engineering, to solve the problem of ATE in opinionated documents. The proposed architecture consists of a BLSTM-CRF classifier that uses the part-of-speech tag (POS tags) as an additional feature, along with a BLSTM encoder with an attention mechanism to allow the incorporation of another relevant feature: the grammatical relations between words. The experiments show that the proposed architecture achieves promising results with minimal feature engineering comparing to the state-of-the-art solutions.
Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Connections to Evolvability

In this paper we revisit some classic problems on classification under misspecification. In particular, we study the problem of learning halfspaces under Massart noise with rate $\eta$. In a recent work, Diakonikolas, Goulekakis, and Tzamos resolved a long-standing problem by giving the first efficient algorithm for learning to accuracy $\eta + \epsilon$ for any $\epsilon > 0$. However, their algorithm outputs a complicated hypothesis, which partitions space into $\text{poly}(d,1/\epsilon)$ regions. Here we give a much simpler algorithm and in the process resolve a number of outstanding open questions: (1) We give the first proper learner for Massart halfspaces that achieves $\eta + \epsilon$. We also give improved bounds on the sample complexity achievable by polynomial time algorithms. (2) Based on (1), we develop a blackbox knowledge distillation procedure to convert an arbitrarily complex classifier to an equally good proper classifier. (3) By leveraging a simple but overlooked connection to evolvability, we show any SQ algorithm requires super-polynomially many queries to achieve $\mathsf{OPT} + \epsilon$. Moreover we study generalized linear models where $\mathbb{E}[Y|\mathbf{X}] = \sigma(\langle \mathbf{w}^*, \mathbf{X}\rangle)$ for any odd, monotone, and Lipschitz function $\sigma$. This family includes the previously mentioned halfspace models as a special case, but is much richer and includes other fundamental models like logistic regression. We introduce a challenging new corruption model that generalizes Massart noise, and give a general algorithm for learning in this setting. Our algorithms are based on a small set of core recipes for learning to classify in the presence of misspecification. Finally we study our algorithm for learning halfspaces under Massart noise empirically and find that it exhibits some appealing fairness properties.
Defense for Black-box Attacks on Anti-spoofing Models by Self-Supervised Learning

High-performance anti-spoofing models for automatic speaker verification (ASV), have been widely used to protect ASV by identifying and filtering spoofing audio that is deliberately generated by text-to-speech, voice conversion, audio replay, etc. However, it has been shown that high-performance anti-spoofing models are vulnerable to adversarial attacks. Adversarial attacks, that are indistinguishable from original data but result in the incorrect predictions, are dangerous for anti-spoofing models and not in dispute we should detect them at any cost. To explore this issue, we proposed to employ Mockingjay, a self-supervised learning based model, to protect anti-spoofing models against adversarial attacks in the black-box scenario. Self-supervised learning models are effective in improving downstream task performance like phone classification or ASR. However, their effect in defense for adversarial attacks has not been explored yet. In this work, we explore the robustness of self-supervised learned high-level representations by using them in the defense against adversarial attacks. A layerwise noise to signal ratio (LNSR) is proposed to quantize and measure the effectiveness of deep models in countering adversarial noise. Experimental results on the ASVspoof 2019 dataset demonstrate that high-level representations extracted by Mockingjay can prevent the transferability of adversarial examples, and successfully counter black-box attacks.
Creating and Implementing an Effective and Deterrent National Cyber Security Strategy

This paper presents a new national cyber security strategy (NCSS) covering the deterrence perspective from creation to implementation. With the aim of responding to and ensuring cyber security effectively, studies on which pathways should be followed and what methods should be used to develop, create, and implement a NCSS are being conducted in Turkey, as in all countries. In this context, by explaining the importance of cyber power, the need for cyber power to be considered as one of the elements of national power and the importance of providing security against cyber attacks with deterrence by cyber power are discussed, while a new and integrated approach for the creation and implementation of a NCSS and an authoritarian organizational structure responsible for this strategy is proposed. It can be concluded that the proposed effective and deterrent NCSS model and approaches might help to efficiently and effectively handle these issues for better management, control, and auditing for cyber security issues; provide new concepts for cyber security issues supported by cyber power and deterrence on this issue in the world; introduce integrated approach to cyber security strategies and policies on the stages of developing and implementing a NCSS; bring a number of issues to support cyber security and defence in different perspectives; and achieve a high degree of success, especially in terms of effectiveness and existing deterrence strategies and action plans of basic structure, with the proposed approach. Finally, it is expected that the proposed strategy, steps, and suggestions might help to improve cyber security issues and national strategies in near future to secure national assessment more than ever with a powerful concept of deterrence. In addition, this approach, which has been put forward for effective implementation of cyber security by ensuring better management, control, and supervision, can be easily used by all other countries.
A Global Inhomogeneous Intensity Clustering- (GINC-) Based Active Contour Model for Image Segmentation and Bias Correction

Image segmentation is still an open problem especially when intensities of the objects of interest are overlapped due to the presence of intensity inhomogeneities. A bias correction embedded level set model is proposed in this paper where inhomogeneities are estimated by orthogonal primary functions. First, an inhomogeneous intensity clustering energy is defined based on global distribution characteristics of the image intensities, and membership functions of the clusters described by the level set function are then introduced to define the data term energy of the proposed model. Second, a regularization term and an arc length term are also included to regularize the level set function and smooth its zero-level set contour, respectively. Third, the proposed model is extended to multichannel and multiphase patterns to segment colorful images and images with multiple objects, respectively. Experimental results and comparison with relevant models demonstrate the advantages of the proposed model in terms of bias correction and segmentation accuracy on widely used synthetic and real images and the BrainWeb and the IBSR image repositories.
Explicitly Modeled Attention Maps for Image Classification

Self-attention networks have shown remarkable progress in computer vision tasks such as image classification. The main benefit of the self-attention mechanism is the ability to capture long-range feature interactions in attention-maps. However, the computation of attention-maps requires a learnable key, query, and positional encoding, whose usage is often not intuitive and computationally expensive. To mitigate this problem, we propose a novel self-attention module with explicitly modeled attention-maps using only a single learnable parameter for low computational overhead. The design of explicitly modeled attention-maps using geometric prior is based on the observation that the spatial context for a given pixel within an image is mostly dominated by its neighbors, while more distant pixels have a minor contribution. Concretely, the attention-maps are parametrized via simple functions (e.g., Gaussian kernel) with a learnable radius, which is modeled independently of the input content. Our evaluation shows that our method achieves an accuracy improvement of up to 2.2% over the ResNet-baselines in ImageNet ILSVRC and outperforms other self-attention methods such as AA-ResNet152 in accuracy by 0.9% with 6.4% fewer parameters and 6.7% fewer GFLOPs. This result empirically indicates the value of incorporating geometric prior into self-attention mechanism when applied in image classification.
Discrete Latent Variable Representations for Low-Resource Text Classification

While much work on deep latent variable models of text uses continuous latent variables, discrete latent variables are interesting because they are more interpretable and typically more space efficient.We consider several approaches to learning discrete latent variable models for text in the case where exact marginalization over these variables is intractable.We compare the performance of the learned representations as features for lowresource document and sentence classification.Our best models outperform the previous best reported results with continuous representations in these low-resource settings, while learning significantly more compressed representations.Interestingly, we find that an amortized variant of Hard EM performs particularly well in the lowest-resource regimes. 1
Learning-Based Affine Registration of Histological Images

The use of different stains for histological sample preparation reveals distinct tissue properties and may result in a more accurate diagnosis. However, as a result of the staining process, the tissue slides are being deformed and registration is required before further processing. The importance of this problem led to organizing an open challenge named Automatic Non-rigid Histological Image Registration Challenge (ANHIR), organized jointly with the IEEE ISBI 2019 conference. The challenge organizers provided several hundred image pairs and a server-side evaluation platform. One of the most difficult sub-problems for the challenge participants was to find an initial, global transform, before attempting to calculate the final, non-rigid deformation field. This article solves the problem by proposing a deep network trained in an unsupervised way with a good generalization. We propose a method that works well for images with different resolutions, aspect ratios, without the necessity to perform image padding, while maintaining a low number of network parameters and fast forward pass time. The proposed method is orders of magnitude faster than the classical approach based on the iterative similarity metric optimization or computer vision descriptors. The success rate is above 98% for both the training set and the evaluation set. We make both the training and inference code freely available.
Better depth-width trade-offs for neural networks through the lens of dynamical systems

The expressivity of neural networks as a function of their depth, width and type of activation units has been an important question in deep learning theory. Recently, depth separation results for ReLU networks were obtained via a new connection with dynamical systems, using a generalized notion of fixed points of a continuous map $f$, called periodic points. In this work, we strengthen the connection with dynamical systems and we improve the existing width lower bounds along several aspects. Our first main result is period-specific width lower bounds that hold under the stronger notion of $L^1$-approximation error, instead of the weaker classification error. Our second contribution is that we provide sharper width lower bounds, still yielding meaningful exponential depth-width separations, in regimes where previous results wouldn't apply. A byproduct of our results is that there exists a universal constant characterizing the depth-width trade-offs, as long as $f$ has odd periods. Technically, our results follow by unveiling a tighter connection between the following three quantities of a given function: its period, its Lipschitz constant and the growth rate of the number of oscillations arising under compositions of the function $f$ with itself.
Impact of Information System and Blockchain on Organizational Learning Effectiveness

The focus of this paper is to investigate the impact of information systems and blockchain technology on organizational learning effectiveness. Data was collected from the information system faculty and administrative staff of two different universities in Pakistan and a qualitative approach was used to achieve the objectives of the study. The paper discovered why and how faculty and administrators show resistance or exert influences or assist information system and blockchain initiatives in organizational learning effectiveness and sustainability. A qualitative approach was used in the study to comprehend the recurring phenomenon. The research finding demands the support of the senior faculty members, administrators, and most importantly, the leadership, in the design, orientation, and implementation of information systems and blockchain for organizational learning effectiveness.
Deep Learning and Machine Learning in Robotics [From the Guest Editors]

Deep learning has gone through massive growth in recent years. In many fields—computer vision, speech recognition, machine translation, game playing, and others—deep learning has brought unprecedented progress and become the method of choice. Will the same happen in robotics and automation? In a sense, it is already happening. Today, deep learning is often the most common keyword for work presented at major robotics conferences. At the same time, robots, as physical systems, pose unique challenges for deep learning in terms of sample efficiency and safety in real-world robot applications. With robots, data are abundant, but labels are sparse and expensive to acquire. Reinforcement learning in principle does not require data labeling but does require a significant number of iterations on real robots. Transferring the capabilities learned in simulation to real robots and collecting sufficient data for practical robot applications both present major challenges. Further, mistakes by robot learning systems are often much more costly than those by their counterparts in the virtual world. These mistakes may cause irreversible damage to robot hardware or, even worse, loss of human lives. Safety is thus paramount for robot learning systems.
MC-BERT: Efficient Language Pre-Training via a Meta Controller

Pre-trained contextual representations (e.g., BERT) have become the foundation to achieve state-of-the-art results on many NLP tasks. However, large-scale pre-training is computationally expensive. ELECTRA, an early attempt to accelerate pre-training, trains a discriminative model that predicts whether each input token was replaced by a generator. Our studies reveal that ELECTRA's success is mainly due to its reduced complexity of the pre-training task: the binary classification (replaced token detection) is more efficient to learn than the generation task (masked language modeling). However, such a simplified task is less semantically informative. To achieve better efficiency and effectiveness, we propose a novel meta-learning framework, MC-BERT. The pre-training task is a multi-choice cloze test with a reject option, where a meta controller network provides training input and candidates. Results over GLUE natural language understanding benchmark demonstrate that our proposed method is both efficient and effective: it outperforms baselines on GLUE semantic tasks given the same computational budget.
Design and Structure of The Juman++ Morphological Analyzer Toolkit

An NLP tool is practical when it is fast in addition to having high accuracy. We describe the architecture and the used methods to achieve 250× analysis speed improvement on the Juman++ morphological analyzer together with slight accuracy improvements. This information should be useful for implementors of high-performance NLP and machine-learning based software.