OpenRaiser/Intern-Atlas huggingface dataset:
Compositional Processing Emerges in Neural Networks Solving Math Problems
Venue: arXiv cs.LG 2021
Year: 2021

A longstanding question in cognitive science concerns the learning mechanisms underlying compositionality in human cognition. Humans can infer the structured relationships (e.g., grammatical rules) implicit in their sensory observations (e.g., auditory speech), and use this knowledge to guide the composition of simpler meanings into complex wholes. Recent progress in artificial neural networks has shown that when large models are trained on enough linguistic data, grammatical structure emerges in their representations. We extend this work to the domain of mathematical reasoning, where it is possible to formulate precise hypotheses about how meanings (e.g., the quantities corresponding to numerals) should be composed according to structured rules (e.g., order of operations). Our work shows that neural networks are not only able to infer something about the structured relationships implicit in their training data, but can also deploy this knowledge to guide the composition of individual meanings into composite wholes.
Speed Scaling On Parallel Servers with MapReduce Type Precedence Constraints
Venue: arXiv cs.DS 2021
Year: 2021

A multiple server setting is considered, where each server has tunable speed, and increasing the speed incurs an energy cost. Jobs arrive to a single queue, and each job has two types of sub-tasks, map and reduce, and a {\bf precedence} constraint among them: any reduce task of a job can only be processed once all the map tasks of the job have been completed. In addition to the scheduling problem, i.e., which task to execute on which server, with tunable speed, an additional decision variable is the choice of speed for each server, so as to minimize a linear combination of the sum of the flow times of jobs/tasks and the total energy cost. The precedence constraints present new challenges for the speed scaling problem with multiple servers, namely that the number of tasks that can be executed at any time may be small but the total number of outstanding tasks might be quite large. We present simple speed scaling algorithms that are shown to have competitive ratios, that depend on the power cost function, and/or the ratio of the size of the largest task and the shortest reduce task, but not on the number of jobs, or the number of servers.
On the Secrecy Capacity of 2-user Gaussian Z-Interference Channel with Shared Key
Venue: arXiv cs.IT 2021
Year: 2021

In this paper, the role of secret key with finite rate is studied to enhance the secrecy performance of the system when users are operating in interference limited scenarios. To address this problem, a 2-user Gaussian Z-IC with secrecy constraint at the receiver is considered. One of the fundamental problems here is how to use the secret key as a part of the encoding process. The paper proposes novel achievable schemes, where the schemes differ from each other based on how the key has been used in the encoding process. The first achievable scheme uses one part of the key for one-time pad and remaining part of the key for wiretap coding. The encoding is performed such that the receiver experiencing interference can decode some part of the interference without violating the secrecy constraint. As a special case of the derived result, one can obtain the secrecy rate region when the key is completely used for one-time pad or part of the wiretap coding. The second scheme uses the shared key to encrypt the message using one-time pad and in contrast to the previous case no interference is decoded at the receiver. The paper also derives an outer bound on the sum rate and secrecy rate of the transmitter which causes interference. The main novelty of deriving outer bound lies in the selection of side information provided to the receiver and using the secrecy constraint at the receiver. The derived outer bounds are found to be tight depending on the channel conditions and rate of the key. The scaling behaviour of key rate is also explored for different schemes using the notion of secure GDOF. The optimality of different schemes is characterized for some specific cases. The developed results show the importance of key rate splitting in enhancing the secrecy performance of the system when users are operating under interference limited environment.
Revisiting Data Compression in Column-Stores
Venue: arXiv cs.DB 2021
Year: 2021

Data compression is widely used in contemporary column-oriented DBMSes to lower space usage and to speed up query processing. Pioneering systems have introduced compression to tackle the disk bandwidth bottleneck by trading CPU processing power for it. The main issue of this is a trade-off between the compression ratio and the decompression CPU cost. Existing results state that light-weight compression with small decompression costs outperforms heavy-weight compression schemes in column-stores. However, since the time these results were obtained, CPU, RAM, and disk performance have advanced considerably. Moreover, novel compression algorithms have emerged. In this paper, we revisit the problem of compression in disk-based column-stores. More precisely, we study the I/O-RAM compression scheme which implies that there are two types of pages of different size: disk pages (compressed) and in-memory pages (uncompressed). In this scheme, the buffer manager is responsible for decompressing pages as soon as they arrive from disk. This scheme is rather popular as it is easy to implement: several modern column and row-stores use it. We pose and address the following research questions: 1) Are heavy-weight compression schemes still inappropriate for disk-based column-stores?, 2) Are new light-weight compression algorithms better than the old ones?, 3) Is there a need for SIMD-employing decompression algorithms in case of a disk-based system? We study these questions experimentally using a columnar query engine and Star Schema Benchmark.
The State of AI Ethics Report (Volume 4)
Venue: arXiv cs.CY 2021
Year: 2021

The 4th edition of the Montreal AI Ethics Institute's The State of AI Ethics captures the most relevant developments in the field of AI Ethics since January 2021. This report aims to help anyone, from machine learning experts to human rights activists and policymakers, quickly digest and understand the ever-changing developments in the field. Through research and article summaries, as well as expert commentary, this report distills the research and reporting surrounding various domains related to the ethics of AI, with a particular focus on four key themes: Ethical AI, Fairness & Justice, Humans & Tech, and Privacy. In addition, The State of AI Ethics includes exclusive content written by world-class AI Ethics experts from universities, research institutes, consulting firms, and governments. Opening the report is a long-form piece by Edward Higgs (Professor of History, University of Essex) titled "AI and the Face: A Historian's View." In it, Higgs examines the unscientific history of facial analysis and how AI might be repeating some of those mistakes at scale. The report also features chapter introductions by Alexa Hagerty (Anthropologist, University of Cambridge), Marianna Ganapini (Faculty Director, Montreal AI Ethics Institute), Deborah G. Johnson (Emeritus Professor, Engineering and Society, University of Virginia), and Soraj Hongladarom (Professor of Philosophy and Director, Center for Science, Technology and Society, Chulalongkorn University in Bangkok). This report should be used not only as a point of reference and insight on the latest thinking in the field of AI Ethics, but should also be used as a tool for introspection as we aim to foster a more nuanced conversation regarding the impacts of AI on the world.
Combining GCN and Transformer for Chinese Grammatical Error Detection
Venue: arXiv cs.CL 2021
Year: 2021

This paper describes our system at NLPTEA-2020 Task: Chinese Grammatical Error Diagnosis (CGED). The goal of CGED is to diagnose four types of grammatical errors: word selection (S), redundant words (R), missing words (M), and disordered words (W). The automatic CGED system contains two parts including error detection and error correction and our system is designed to solve the error detection problem. Our system is built on three models: 1) a BERT-based model leveraging syntactic information; 2) a BERT-based model leveraging contextual embeddings; 3) a lexicon-based graph neural network leveraging lexical information. We also design an ensemble mechanism to improve the single model's performance. Finally, our system achieves the highest F1 scores at detection level and identification level among all teams participating in the CGED 2020 task.
Learn Fine-grained Adaptive Loss for Multiple Anatomical Landmark Detection in Medical Images
Venue: arXiv cs.CV 2021
Year: 2021

Automatic and accurate detection of anatomical landmarks is an essential operation in medical image analysis with a multitude of applications. Recent deep learning methods have improved results by directly encoding the appearance of the captured anatomy with the likelihood maps (i.e., heatmaps). However, most current solutions overlook another essence of heatmap regression, the objective metric for regressing target heatmaps and rely on hand-crafted heuristics to set the target precision, thus being usually cumbersome and task-specific. In this paper, we propose a novel learning-to-learn framework for landmark detection to optimize the neural network and the target precision simultaneously. The pivot of this work is to leverage the reinforcement learning (RL) framework to search objective metrics for regressing multiple heatmaps dynamically during the training process, thus avoiding setting problem-specific target precision. We also introduce an early-stop strategy for active termination of the RL agent's interaction that adapts the optimal precision for separate targets considering exploration-exploitation tradeoffs. This approach shows better stability in training and improved localization accuracy in inference. Extensive experimental results on two different applications of landmark localization: 1) our in-house prenatal ultrasound (US) dataset and 2) the publicly available dataset of cephalometric X-Ray landmark detection, demonstrate the effectiveness of our proposed method. Our proposed framework is general and shows the potential to improve the efficiency of anatomical landmark detection.
Robustness and stability of enterprise intranet social networks: The impact of moderators
Venue: arXiv cs.SI 2021
Year: 2021

In this study, we tested the robustness of three communication networks extracted from the online forums included in the intranet platforms of three large companies. For each company we analyzed the communication among employees both in terms of network structure and content (language used). Over a period of eight months, we analyzed more than 52,000 messages posted by approximately 12,000 employees. Specifically, we tested the network robustness and the stability of a set of structural and semantic metrics, while applying several different node removal strategies. We removed the forum moderators, the spammers, the overly connected nodes and the nodes lying at the network periphery, also testing different combinations of these selections. Results indicate that removing spammers and very peripheral nodes can be a relatively low impact strategy in this context; accordingly, it could be used to clean the noise generated by these types of social actor and to reduce the computation complexity of the analysis. On the other hand, the removal of moderators seems to have a significant impact on the network connectivity and the shared content. The most affected variables are closeness centrality and contribution index. We also found that the removal of overly connected nodes can significantly change the network structure. Lastly, we compared the behavior of moderators with the other users, finding distinctive characteristics by which moderators can be identified when their list is unknown. Our findings can help online community managers to understand the role of moderators within intranet forums and can be useful for social network analysts who are interested in evaluating the effects of graph simplification techniques.
Procedural animations in interactive art experiences -- A state of the art review
Venue: arXiv cs.HC 2021
Year: 2021

The state of the art review broadly oversees the use of novel research utilized in the creation of virtual environments applied in interactive art experiences, with a specific focus on the application of procedural animation in spatially augmented reality (SAR) exhibitions. These art exhibitions frequently combine sensory displays that appeal, replace, and augment the visual, auditory and touch or haptic senses. We analyze and break down art-technology related innovations in the last three years, and thoroughly identify the most recent and vibrant applications of interactive art experiences in the review of numerous installation applications, studies, and events. Display mediums such as virtual reality, augmented reality, mixed reality, and robotics are overviewed in the context of art experiences such as visual art museums, park or historic site tours, live concerts, and theatre. We explore research and extrapolate how recent innovations can lead to different applications that will be seen in the future.
Hunter in the Dark: Discover Anomalous Network Activity Using Deep Ensemble Network
Venue: arXiv cs.CR 2021
Year: 2021

Machine learning (ML)-based intrusion detection systems (IDSs) play a critical role in discovering unknown threats in a large-scale cyberspace. They have been adopted as a mainstream hunting method in many organizations, such as financial institutes, manufacturing companies and government agencies. However, existing designs achieve a high threat detection performance at the cost of a large number of false alarms, leading to alert fatigue. To tackle this issue, in this paper, we propose a neural-network-based defense mechanism named DarkHunter. DarkHunter incorporates both supervised learning and unsupervised learning in the design. It uses a deep ensemble network (trained through supervised learning) to detect anomalous network activities and exploits an unsupervised learning-based scheme to trim off mis-detection results. For each detected threat, DarkHunter can trace to its source and present the threat in its original traffic format. Our evaluations, based on the UNSW-NB15 dataset, show that DarkHunter outperforms the existing ML-based IDSs and is able to achieve a high detection accuracy while keeping a low false positive rate.
Approximation Algorithms For The Euclidean Dispersion Problems
Venue: arXiv cs.CG 2021
Year: 2021

In this article, we consider the Euclidean dispersion problems. Let $P=\{p_{1}, p_{2}, \ldots, p_{n}\}$ be a set of $n$ points in $\mathbb{R}^2$. For each point $p \in P$ and $S \subseteq P$, we define $cost_γ(p,S)$ as the sum of Euclidean distance from $p$ to the nearest $γ$ point in $S \setminus \{p\}$. We define $cost_γ(S)=\min_{p \in S}\{cost_γ(p,S)\}$ for $S \subseteq P$. In the $γ$-dispersion problem, a set $P$ of $n$ points in $\mathbb{R}^2$ and a positive integer $k \in [γ+1,n]$ are given. The objective is to find a subset $S\subseteq P$ of size $k$ such that $cost_γ(S)$ is maximized. We consider both $2$-dispersion and $1$-dispersion problem in $\mathbb{R}^2$. Along with these, we also consider $2$-dispersion problem when points are placed on a line. In this paper, we propose a simple polynomial time $(2\sqrt 3 + ε)$-factor approximation algorithm for the $2$-dispersion problem, for any $ε> 0$, which is an improvement over the best known approximation factor $4\sqrt3$ [Amano, K. and Nakano, S. I., An approximation algorithm for the $2$-dispersion problem, IEICE Transactions on Information and Systems, Vol. 103(3), pp. 506-508, 2020]. Next, we develop a common framework for designing an approximation algorithm for the Euclidean dispersion problem. With this common framework, we improve the approximation factor to $2\sqrt 3$ for the $2$-dispersion problem in $\mathbb{R}^2$. Using the same framework, we propose a polynomial time algorithm, which returns an optimal solution for the $2$-dispersion problem when points are placed on a line. Moreover, to show the effectiveness of the framework, we also propose a $2$-factor approximation algorithm for the $1$-dispersion problem in $\mathbb{R}^2$.
DumbleDR: Predicting User Preferences of Dimensionality Reduction Projection Quality
Venue: arXiv cs.LG 2021
Year: 2021

A plethora of dimensionality reduction techniques have emerged over the past decades, leaving researchers and analysts with a wide variety of choices for reducing their data, all the more so given some techniques come with additional parametrization (e.g. t-SNE, UMAP, etc.). Recent studies are showing that people often use dimensionality reduction as a black-box regardless of the specific properties the method itself preserves. Hence, evaluating and comparing 2D projections is usually qualitatively decided, by setting projections side-by-side and letting human judgment decide which projection is the best. In this work, we propose a quantitative way of evaluating projections, that nonetheless places human perception at the center. We run a comparative study, where we ask people to select 'good' and 'misleading' views between scatterplots of low-level projections of image datasets, simulating the way people usually select projections. We use the study data as labels for a set of quality metrics whose purpose is to discover and quantify what exactly people are looking for when deciding between projections. With this proxy for human judgments, we use it to rank projections on new datasets, explain why they are relevant, and quantify the degree of subjectivity in projections selected.
Informatiekunde -- Visie 2003
Venue: arXiv cs.CY 2021
Year: 2021

This document discusses (in Dutch) the vision underlying the business informatics (Informatiekunde) curriculum and research programme at the Nijmegen Institute for Informatics and Information Science (NIII). The ultimate aim of this document is to provide a 'repository' with regard to these visions, and a basis for the specific structure of the program's curriculum and research plans. Since business informatics is a relatively new area for teaching and research at NIII, the current (2003) version of this document primarily focuses on the educational perspective. It is to be expected that in the coming years, updates to this document will also pay more attention to business informatics research. The fact that this document can be updated annually does not mean, however, that we expect an annual change of course. The ambition with regard to the stability of what is described in this document is 5 to 6 years. In the current version, this applies specifically to the vision of the information science study program. The research part of this document will have to be fleshed out even more specifically in the coming years.
DeepDebug: Fixing Python Bugs Using Stack Traces, Backtranslation, and Code Skeletons
Venue: arXiv cs.SE 2021
Year: 2021

The joint task of bug localization and program repair is an integral part of the software development process. In this work we present DeepDebug, an approach to automated debugging using large, pretrained transformers. We begin by training a bug-creation model on reversed commit data for the purpose of generating synthetic bugs. We apply these synthetic bugs toward two ends. First, we directly train a backtranslation model on all functions from 200K repositories. Next, we focus on 10K repositories for which we can execute tests, and create buggy versions of all functions in those repositories that are covered by passing tests. This provides us with rich debugging information such as stack traces and print statements, which we use to finetune our model which was pretrained on raw source code. Finally, we strengthen all our models by expanding the context window beyond the buggy function itself, and adding a skeleton consisting of that function's parent class, imports, signatures, docstrings, and method bodies, in order of priority. On the QuixBugs benchmark, we increase the total number of fixes found by over 50%, while also decreasing the false positive rate from 35% to 5% and decreasing the timeout from six hours to one minute. On our own benchmark of executable tests, our model fixes 68% of all bugs on its first attempt without using traces, and after adding traces it fixes 75% on first attempt. We will open-source our framework and validation set for evaluating on executable tests.
Non-cumulative measures of researcher citation impact
Venue: arXiv cs.DL 2021
Year: 2021

The most commonly used publication metrics for individual researchers are the the total number of publications, the total number of citations, and Hirsch's $h$-index. Each of these is cumulative, and hence increases throughout a researcher's career, making it less suitable for evaluation of junior researchers or assessing recent impact. Most other author-level measures in the literature share this cumulative property. By contrast, we aim to study non-cumulative measures that answer the question "in terms of citation impact, what have you done lately?" We single out six measures from the rather sparse literature, including Hirsch's $m$-index, a time-scaled version of the $h$-index. We introduce new measures based on the idea of "citation acceleration". After presenting several axioms for non-cumulative measures, we conclude that one of our new measures has much better theoretical justification. We present a small-scale study of its performance on real data and conclude that it shows substantial promise for future use.
Geographic Question Answering: Challenges, Uniqueness, Classification, and Future Directions
Venue: arXiv cs.CL 2021
Year: 2021

As an important part of Artificial Intelligence (AI), Question Answering (QA) aims at generating answers to questions phrased in natural language. While there has been substantial progress in open-domain question answering, QA systems are still struggling to answer questions which involve geographic entities or concepts and that require spatial operations. In this paper, we discuss the problem of geographic question answering (GeoQA). We first investigate the reasons why geographic questions are difficult to answer by analyzing challenges of geographic questions. We discuss the uniqueness of geographic questions compared to general QA. Then we review existing work on GeoQA and classify them by the types of questions they can address. Based on this survey, we provide a generic classification framework for geographic questions. Finally, we conclude our work by pointing out unique future research directions for GeoQA.
Speech & Song Emotion Recognition Using Multilayer Perceptron and Standard Vector Machine
Venue: arXiv cs.SD 2021
Year: 2021

Herein, we have compared the performance of SVM and MLP in emotion recognition using speech and song channels of the RAVDESS dataset. We have undertaken a journey to extract various audio features, identify optimal scaling strategy and hyperparameter for our models. To increase sample size, we have performed audio data augmentation and addressed data imbalance using SMOTE. Our data indicate that optimised SVM outperforms MLP with an accuracy of 82 compared to 75%. Following data augmentation, the performance of both algorithms was identical at ~79%, however, overfitting was evident for the SVM. Our final exploration indicated that the performance of both SVM and MLP were similar in which both resulted in lower accuracy for the speech channel compared to the song channel. Our findings suggest that both SVM and MLP are powerful classifiers for emotion recognition in a vocal-dependent manner.