Reviewerly/exHarmony huggingface dataset:
Project Debater APIs: Decomposing the AI Grand Challenge

Project Debater was revealed in 2019 as the first AI system that can debate human experts on complex topics. Engaging in a live debate requires a diverse set of skills, and Project Debater has been developed accordingly as a collection of components, each designed to perform a specific subtask. Project Debater APIs provide access to many of these capabilities, as well as to more recently developed ones. This diverse set of web services, publicly available for academic use, includes core NLP services, argument mining and analysis capabilities, and higher-level services for content summarization. We describe these APIs and their performance, and demonstrate how they can be used for building practical solutions. In particular, we will focus on Key Point Analysis, a novel technology that identifies the main points and their prevalence in a collection of texts such as survey responses and user reviews.
The Limitations of Large Width in Neural Networks: A Deep Gaussian Process Perspective

Large width limits have been a recent focus of deep learning research: modulo computational practicalities, do wider networks outperform narrower ones? Answering this question has been challenging, as conventional networks gain representational power with width, potentially masking any negative effects. Our analysis in this paper decouples capacity and width via the generalization of neural networks to Deep Gaussian Processes (Deep GP), a class of hierarchical models that subsume neural nets. In doing so, we aim to understand how width affects standard neural networks once they have sufficient capacity for a given modeling task. Our theoretical and empirical results on Deep GP suggest that large width is generally detrimental to hierarchical models. Surprisingly, we prove that even nonparametric Deep GP converge to Gaussian processes, effectively becoming shallower without any increase in representational power. The posterior, which corresponds to a mixture of data-adaptable basis functions, becomes less data-dependent with width. Our tail analysis demonstrates that width and depth have opposite effects: depth accentuates a model's non-Gaussianity, while width makes models increasingly Gaussian. We find there is a sweet spot that maximizes test set performance before the limiting GP behavior prevents adaptability, occurring at width = 1 or width = 2 for nonparametric Deep GP. These results make strong predictions about the same phenomenon in conventional neural networks: we show empirically that many neural network architectures need 10 - 500 hidden units for sufficient capacity - depending on the dataset - but further width degrades test performance.
Assessing Organisational Information Security Culture Among Workforce in Universities: A Case of Namibia

Information security culture is one of the crucial elements that need to be established to protect organisational assets. Many organisations usually employ technical approach to protect business information. However, complete information security needs to consider the human, processes and procedures. Technical controls, processes and procedures need humans to implement and manage them effectively if information security is to be achieved. Core to human security is behaviour and culture. Technical techniques need to be supported by adopting a proper information security culture among employees. Introducing a culture whereby information that must be protected and governed by the entire workforce at all times in agreement with organisational policy and regulatory requirements is of paramount importance. University communities are no exception to this subject matter. The main objective of this study was to evaluate organisational information security culture amongst a university community. A case study was carried out at a university in Namibia where data to establish the state of security culture was collected using surveys and interviews. Qualitative data analysis techniques were applied. Some of the findings highlighted lack of knowledge and understanding of the institution information security policies, lack of information security awareness and lack of executive involvement in information security policy formulation and implementation. It was also noted that there was no compliance to existing information security policies as only a few individuals were aware of the existence of organisational security policies. In conclusion, ways to improve organisational information security culture were proposed.
SYSML: StYlometry with Structure and Multitask Learning: Implications for Darknet Forum Migrant Analysis.

Darknet market forums are frequently used to exchange illegal goods and services between parties who use encryption to conceal their identities. The Tor network is used to host these markets, which guarantees additional anonymization from IP and location tracking, making it challenging to link across malicious users using multiple accounts (sybils). Additionally, users migrate to new forums when one is closed further increasing the difficulty of linking users across multiple forums. We develop a novel stylometry-based multitask learning approach for natural language and model interactions using graph embeddings to construct low-dimensional representations of short episodes of user activity for authorship attribution. We provide a comprehensive evaluation of our methods across four different darknet forums demonstrating its efficacy over the state-of-the-art, with a lift of up to 2.5X on Mean Retrieval Rank and 2X on Recall@10.
Reducing Discontinuous to Continuous Parsing with Pointer Network Reordering

Discontinuous constituent parsers have always lagged behind continuous approaches in terms of accuracy and speed, as the presence of constituents with discontinuous yield introduces extra complexity to the task. However, a discontinuous tree can be converted into a continuous variant by reordering tokens. Based on that, we propose to reduce discontinuous parsing to a continuous problem, which can then be directly solved by any off-the-shelf continuous parser. To that end, we develop a Pointer Network capable of accurately generating the continuous token arrangement for a given input sentence and define a bijective function to recover the original order. Experiments on the main benchmarks with two continuous parsers prove that our approach is on par in accuracy with purely discontinuous state-of-the-art algorithms, but considerably faster.
Cloud-Assisted Hybrid Rendering for Thin-Client Games and VR Applications

We introduce a novel distributed rendering approach to generate high-quality graphics in thin-client games and VR applications. Many mobile devices have limited computational power to achieve ray tracing in real-time. Hence, hardware-accelerated cloud servers can perform ray tracing instead and have their output streamed to clients in remote rendering. Applying the approach of distributed hybrid rendering, we leverage the computational capabilities of both the thin client and powerful server by performing rasterization locally while offloading ray tracing to the server. With advancements in 5G technology, the server and client can communicate effectively over the network and work together to produce a high-quality output while maintaining interactive frame rates. Our approach can achieve better visuals as compared to local rendering but faster performance as compared to remote rendering.
DA-LMR: A Robust Lane Markings Representation for Data Association Methods

While complete localization approaches are widely studied in the literature, their data association and data representation subprocesses usually go unnoticed. However, both are a key part of the final pose estimation. In this work, we present DA-LMR (Delta-Angle Lane Markings Representation), a robust data representation in the context of localization approaches. We propose a representation of lane markings that encodes how a curve changes in each point and includes this information in an additional dimension, thus providing a more detailed geometric structure description of the data. We also propose DC-SAC (Distance-Compatible Sample Consensus), a data association method. This is a heuristic version of RANSAC that dramatically reduces the hypothesis space by distance compatibility restrictions. We compare the presented methods with some state-of-the-art data representation and data association approaches in different noisy scenarios. The DA-LMR and DC-SAC produce the most promising combination among those compared, reaching 98.1% in precision and 99.7% in recall for noisy data with 0.5m of standard deviation.
PortaSpeech: Portable and High-Quality Generative Text-to-Speech

Non-autoregressive text-to-speech (NAR-TTS) models such as FastSpeech 2 and Glow-TTS can synthesize high-quality speech from the given text in parallel. After analyzing two kinds of generative NAR-TTS models (VAE and normalizing flow), we find that: VAE is good at capturing the long-range semantics features (e.g., prosody) even with small model size but suffers from blurry and unnatural results; and normalizing flow is good at reconstructing the frequency bin-wise details but performs poorly when the number of model parameters is limited. Inspired by these observations, to generate diverse speech with natural details and rich prosody using a lightweight architecture, we propose PortaSpeech, a portable and high-quality generative text-to-speech model. Specifically, 1) to model both the prosody and mel-spectrogram details accurately, we adopt a lightweight VAE with an enhanced prior followed by a flow-based post-net with strong conditional inputs as the main architecture. 2) To further compress the model size and memory footprint, we introduce the grouped parameter sharing mechanism to the affine coupling layers in the post-net. 3) To improve the expressiveness of synthesized speech and reduce the dependency on accurate fine-grained alignment between text and speech, we propose a linguistic encoder with mixture alignment combining hard inter-word alignment and soft intra-word alignment, which explicitly extracts word-level semantic information. Experimental results show that PortaSpeech outperforms other TTS models in both voice quality and prosody modeling in terms of subjective and objective evaluation metrics, and shows only a slight performance degradation when reducing the model parameters to 6.7M (about 4x model size and 3x runtime memory compression ratio compared with FastSpeech 2). Our extensive ablation studies demonstrate that each design in PortaSpeech is effective.
Improving Reading Comprehension Ability with the Think Pair Share Cooperative Learning Model and Library Utilization of Writing Skills Courses in the Department of Indonesian Language and Literature Semester II at FKIP UMSU Academic Year 2021-2022

Writing skills courses in higher education are learning that requires students to be able to understand reading both in text and context. Students need to be given the opportunity to understand the reading so that they can pour what is on their mind into written form. Ability to read comprehension with the implementation of the Think Pair Share (TPS) Cooperative Learning Model and Library Utilization. Reading Comprehension Ability in Basic Competence of Writing Skills from the results of observations. Cycle I showed the average percentage score of Reading Comprehension in Basic Competence of Writing Skills of 76.19%, the score then increased in cycle II to 84.57% in cycle II. There was an increase in the average percentage score of Reading Comprehension in Basic Competence of Writing Skills from cycle I to cycle II of 8.38%. The results of the average percentage of Reading Comprehension Ability in the Basic Competence of Writing Skills also shows that more than 75% of the indicators of success in Reading Comprehension Basic Competence in Writing Skills have been achieved.
Training Over-parameterized Models with Non-decomposable Objectives

Many modern machine learning applications come with complex and nuanced design goals such as minimizing the worst-case error, satisfying a given precision or recall target, or enforcing group-fairness constraints. Popular techniques for optimizing such non-decomposable objectives reduce the problem into a sequence of cost-sensitive learning tasks, each of which is then solved by re-weighting the training loss with example-specific costs. We point out that the standard approach of re-weighting the loss to incorporate label costs can produce unsatisfactory results when used to train over-parameterized models. As a remedy, we propose new cost-sensitive losses that extend the classical idea of logit adjustment to handle more general cost matrices. Our losses are calibrated, and can be further improved with distilled labels from a teacher model. Through experiments on benchmark image datasets, we showcase the effectiveness of our approach in training ResNet models with common robust and constrained optimization objectives.
Online Adaptation to Label Distribution Shift

Machine learning models often encounter distribution shifts when deployed in the real world. In this paper, we focus on adaptation to label distribution shift in the online setting, where the test-time label distribution is continually changing and the model must dynamically adapt to it without observing the true label. Leveraging a novel analysis, we show that the lack of true label does not hinder estimation of the expected test loss, which enables the reduction of online label shift adaptation to conventional online learning. Informed by this observation, we propose adaptation algorithms inspired by classical online learning techniques such as Follow The Leader (FTL) and Online Gradient Descent (OGD) and derive their regret bounds. We empirically verify our findings under both simulated and real world label distribution shifts and show that OGD is particularly effective and robust to a variety of challenging label shift scenarios.
Learning and Generalization in RNNs

Simple recurrent neural networks (RNNs) and their more advanced cousins LSTMs etc. have been very successful in sequence modeling. Their theoretical understanding, however, is lacking and has not kept pace with the progress for feedforward networks, where a reasonably complete understanding in the special case of highly overparametrized one-hidden-layer networks has emerged. In this paper, we make progress towards remedying this situation by proving that RNNs can learn functions of sequences. In contrast to the previous work that could only deal with functions of sequences that are sums of functions of individual tokens in the sequence, we allow general functions. Conceptually and technically, we introduce new ideas which enable us to extract information from the hidden state of the RNN in our proofs -- addressing a crucial weakness in previous work. We illustrate our results on some regular language recognition problems.
An Uncertainty-Aware Encoder for Aspect Detection.

Aspect detection is a fundamental task in opinion mining. Previous works use seed words either as priors of topic models, as anchors to guide the learning of aspects, or as features of aspect classifiers. This paper presents a novel weakly-supervised method to exploit seed words for aspect detection based on an encoder architecture. The encoder maps segments and aspects into a low-dimensional embedding space. The goal is approximating similarity between segments and aspects in the embedding space and their ground-truth similarity generated from seed words. An objective function is proposed to capture the uncertainty of ground-truth similarity. Our method outperforms previous works on several benchmarks in various domains.
MANAJEMEN E-LEARNING PRA-PANDEMIC 2021

Prior to the pandemic of Covid-19 in 2019, e-learning already implemented to replace or combine with face-to-face learning at universities. Some researchers have argued that e- learning can enhance student learning experiences and therefore, motivated policy makers to design more quality teaching and learning in higher education. This study is a pre-pandemic study of e-learning that helped understand its implementation and aims to describe the implementation in an Islamic Religious Higher Education institution. This research deploys quantitative method with pedagogical, sociological and managerial approach to explore the perceptions of e-learning management among the managers. Paper based questionnaires were distributed to seventeen the heads of the departments at the institution to collect the data. The managers shared their understanding on important factors for enhancing e-learning quality within the information technology infrastructure, content management, design funding, academic policy and learning system. The managers’ views of the e-learning enhancing factors were presented further in this paper.
Statistical Query Lower Bounds for List-Decodable Linear Regression

We study the problem of list-decodable linear regression, where an adversary can corrupt a majority of the examples. Specifically, we are given a set $T$ of labeled examples $(x, y) \in \mathbb{R}^d \times \mathbb{R}$ and a parameter $0< \alpha <1/2$ such that an $\alpha$-fraction of the points in $T$ are i.i.d. samples from a linear regression model with Gaussian covariates, and the remaining $(1-\alpha)$-fraction of the points are drawn from an arbitrary noise distribution. The goal is to output a small list of hypothesis vectors such that at least one of them is close to the target regression vector. Our main result is a Statistical Query (SQ) lower bound of $d^{\mathrm{poly}(1/\alpha)}$ for this problem. Our SQ lower bound qualitatively matches the performance of previously developed algorithms, providing evidence that current upper bounds for this task are nearly best possible.
Multifold behavior of the information transmission by the quantum 3-switch.

We uncover new behaviors of the transmission of information by three quantum channels in superposition of causal orders subject to some level of noise. We find that the transmission can exhibit three different behaviors as the level of noise is varied. This multifold behavior can be explained by the different equivalence classes of quantum switch matrices related to specific combinations of causal orders. We classify these matrices using their characteristic polynomials and matrix invariants, and we calculate analytical expressions for the Holevo information in three representative cases. Our results are a step forward to understand and harness quantum control of causal orders with different levels of noise. We also study the Holevo information as function of a continuous order parameter and analyse transitions at integer values.
A Comprehensive Comparison of Word Embeddings in Event & Entity Coreference Resolution.

Coreference Resolution is an important NLP task and most state-of-the-art methods rely on word embeddings for word representation. However, one issue that has been largely overlooked in literature is that of comparing the performance of different embeddings across and within families. Therefore, we frame our study in the context of Event and Entity Coreference Resolution (EvCR & EnCR), and address two questions : 1) Is there a trade-off between performance (predictive and run-time) and embedding size? 2) How do the embeddings’ performance compare within and across families? Our experiments reveal several interesting findings. First, we observe diminishing returns in performance with respect to embedding size. E.g. a model using solely a character embedding achieves 86% of the performance of the largest model (Elmo, GloVe, Character) while being 1.2% of its size. Second, the larger models using multiple embeddings learns faster despite being slower per epoch. However, it is still slower at test time. Finally, Elmo performs best on both EvCR and EnCR, while GloVe and FastText perform best in EvCR and EnCR respectively.
RoBERTuito: a pre-trained language model for social media text in Spanish.

Since BERT appeared, Transformer language models and transfer learning have become state-of-the-art for Natural Language Understanding tasks. Recently, some works geared towards pre-training, specially-crafted models for particular domains, such as scientific papers, medical documents, and others. In this work, we present RoBERTuito, a pre-trained language model for user-generated content in Spanish. We trained RoBERTuito on 500 million tweets in Spanish. Experiments on a benchmark of 4 tasks involving user-generated text showed that RoBERTuito outperformed other pre-trained language models for Spanish. In order to help further research, we make RoBERTuito publicly available at the HuggingFace model hub.
Multi-Domain Adaptation in Neural Machine Translation Through Multidimensional Tagging

Production NMT systems typically need to serve niche domains that are not covered by adequately large and readily available parallel corpora. As a result, practitioners often fine-tune general purpose models to each of the domains their organisation caters to. The number of domains however can often become large, which in combination with the number of languages that need serving can lead to an unscalable fleet of models to be developed and maintained. We propose Multi Dimensional Tagging, a method for fine-tuning a single NMT model on several domains simultaneously, thus drastically reducing development and maintenance costs. We run experiments where a single MDT model compares favourably to a set of SOTA specialist models, even when evaluated on the domain those baselines have been fine-tuned on. Besides BLEU, we report human evaluation results. MDT models are now live at Booking.com, powering an MT engine that serves millions of translations a day in over 40 different languages.
How to Analyse Chemical Compounds? A Guide to Compositional Data Analysis Demonstrated with Beer Data

In recent years, a lot of analysis was carried out to investigate data on chemical compositions of (non-)alcoholic beverages (whiskey, wine, beer, etc.) and food. However, no study has ever considered the compositional pitfalls inherent in such studies, which may lead to arbitrary results. Two approaches were compared, compositional data analysis (CoDa) and classical statistical analyses. The outcome of compositional data analysis methods provided very well-interpretable results. The method we are proposing is novel in the field of food chemistry, with demonstrated ability to analyse the chemical composition of drinks and food.
A Local Outlier Mining Algorithm Based on Region Segmentation

Regarding high-dimensional heterogeneous data, combined with the existing algorithms' poor mining accuracy and parameter sensitivity, this paper proposes a local outlier mining algorithm based on neighborhood density. Use region segmentation to split high-dimensional data into reasonable sub-regions, reducing the difficulty of processing a large amount of high-dimensional data. The kernel neighborhood density is used to replace the average neighborhood density, so that the density calculation has nothing to do with data heterogeneity. Finally, the neighborhood state and outlier state of the data are further determined on the basis of neighborhood density to improve the accuracy of outlier mining. Through artificial and UCI data set simulation results, it shows that data volume and data dimension are the main factors that affect data outlier mining. The accuracy, coverage, and efficiency of the algorithm proposed in this paper are significantly better than those of the comparison algorithm, and it has better adaptability to different types of data sets.