OpenRaiser/Intern-Atlas huggingface dataset:
Improved skin lesion recognition by a Self-Supervised Curricular Deep Learning approach
Venue: arXiv cs.CV 2021
Year: 2021

State-of-the-art deep learning approaches for skin lesion recognition often require pretraining on larger and more varied datasets, to overcome the generalization limitations derived from the reduced size of the skin lesion imaging datasets. ImageNet is often used as the pretraining dataset, but its transferring potential is hindered by the domain gap between the source dataset and the target dermatoscopic scenario. In this work, we introduce a novel pretraining approach that sequentially trains a series of Self-Supervised Learning pretext tasks and only requires the unlabeled skin lesion imaging data. We present a simple methodology to establish an ordering that defines a pretext task curriculum. For the multi-class skin lesion classification problem, and ISIC-2019 dataset, we provide experimental evidence showing that: i) a model pretrained by a curriculum of pretext tasks outperforms models pretrained by individual pretext tasks, and ii) a model pretrained by the optimal pretext task curriculum outperforms a model pretrained on ImageNet. We demonstrate that this performance gain is related to the fact that the curriculum of pretext tasks better focuses the attention of the final model on the skin lesion. Beyond performance improvement, this strategy allows for a large reduction in the training time with respect to ImageNet pretraining, which is especially advantageous for network architectures tailored for a specific problem.
Hybrid Human-AI Curriculum Development for Personalised Informal Learning Environments
Venue: arXiv cs.CY 2021
Year: 2021

Informal learning procedures have been changing extremely fast over the recent decades not only due to the advent of online learning, but also due to changes in what humans need to learn to meet their various life and career goals. Consequently, online, educational platforms are expected to provide personalized, up-to-date curricula to assist learners. Therefore, in this paper, we propose an Artificial Intelligence (AI) and Crowdsourcing based approach to create and update curricula for individual learners. We show the design of this curriculum development system prototype, in which contributors receive AI-based recommendations to be able to define and update high-level learning goals, skills, and learning topics together with associated learning content. This curriculum development system was also integrated into our personalized online learning platform. To evaluate our prototype we compared experts' opinion with our system's recommendations, and resulted in 89%, 79%, and 93% F1-scores when recommending skills, learning topics, and educational materials respectively. Also, we interviewed eight senior level experts from educational institutions and career consulting organizations. Interviewees agreed that our curriculum development method has high potential to support authoring activities in dynamic, personalized learning environments.
Signaling Design for MIMO-NOMA with Different Security Requirements
Venue: arXiv cs.IT 2021
Year: 2021

Signaling design for secure transmission in two-user multiple-input multiple-output (MIMO) non-orthogonal multiple access (NOMA) networks is investigated in this paper. The base station broadcasts multicast data to all users and also integrates additional services, unicast data targeted to certain users, and confidential data protected against eavesdroppers. We categorize the above MIMO-NOMA with different security requirements into several communication scenarios. The associated problem in each scenario is nonconvex. We propose a unified approach, called the power splitting scheme, for optimizing the rate equations corresponding to the scenarios. The proposed method decomposes the optimization of the secure MIMO-NOMA channel into a set of simpler problems, including multicast, point-to-point, and wiretap MIMO problems, corresponding to the three basic messages: multicast, private/unicast, and confidential messages. We then leverage existing solutions to design signaling for the above problems such that the messages are transmitted with high security and reliability. Numerical results illustrate the efficacy of the proposed covariance matrix design in secure MIMO-NOMA transmission. The proposed method also outperforms existing solutions, when applicable. In the case of no multicast messages, we also reformulate the nonconvex problem into weighted sum rate (WSR) maximization problems by applying the block successive maximization method and generalizing the zero duality gap. The two methods have their advantages and limitations. Power splitting is a general tool that can be applied to the MIMO-NOMA with any combination of the three messages (multicast, private, and confidential) whereas WSR maximization shows greater potential for secure MIMO-NOMA communication without multicasting. In such cases, WSR maximization provides a slightly better rate than the power splitting method.
Fine-grained Multi-Modal Self-Supervised Learning
Venue: arXiv cs.CV 2021
Year: 2021

Multi-Modal Self-Supervised Learning from videos has been shown to improve model's performance on various downstream tasks. However, such Self-Supervised pre-training requires large batch sizes and a large amount of computation resources due to the noise present in the uncurated data. This is partly due to the fact that the prevalent training scheme is trained on coarse-grained setting, in which vectors representing the whole video clips or natural language sentences are used for computing similarity. Such scheme makes training noisy as part of the video clips can be totally not correlated with the other-modality input such as text description. In this paper, we propose a fine-grained multi-modal self-supervised training scheme that computes the similarity between embeddings at finer-scale (such as individual feature map embeddings and embeddings of phrases), and uses attention mechanisms to reduce noisy pairs' weighting in the loss function. We show that with the proposed pre-training scheme, we can train smaller models, with smaller batch-size and much less computational resources to achieve downstream tasks performances comparable to State-Of-The-Art, for tasks including action recognition and text-image retrievals.
Explaining with Examples: Lessons Learned from Crowdsourced Introductory Description of Information Visualizations
Venue: arXiv cs.HC 2021
Year: 2021

Data visualizations have been increasingly used in oral presentations to communicate data patterns to the general public. Clear verbal introductions of visualizations to explain how to interpret the visually encoded information are essential to convey the takeaways and avoid misunderstandings. We contribute a series of studies to investigate how to effectively introduce visualizations to the audience with varying degrees of visualization literacy. We begin with understanding how people are introducing visualizations. We crowdsource 110 introductions of visualizations and categorize them based on their content and structures. From these crowdsourced introductions, we identify different introduction strategies and generate a set of introductions for evaluation. We conduct experiments to systematically compare the effectiveness of different introduction strategies across four visualizations with 1,080 participants. We find that introductions explaining visual encodings with concrete examples are the most effective. Our study provides both qualitative and quantitative insights into how to construct effective verbal introductions of visualizations in presentations, inspiring further research in data storytelling.
Towards Fully Declarative Program Analysis via Source Code Transformation
Venue: arXiv cs.SE 2021
Year: 2021

Advances in logic programming and increasing industrial uptake of Datalog-inspired approaches demonstrate the emerging need to express powerful code analyses more easily. Declarative program analysis frameworks (e.g., using logic programming like Datalog) significantly ease defining analyses compared to imperative implementations. However, the declarative benefits of these frameworks only materialize after parsing and translating source code to generate facts. Fact generation remains a non-declarative precursor to analysis where imperative implementations first parse and interpret program structures (e.g., abstract syntax trees and control-flow graphs). The procedure of fact generation thus remains opaque and difficult for non-experts to understand or modify. We present a new perspective on this analysis workflow by proposing declarative fact generation to ease specification and exploration of lightweight declarative analyses. Our approach demonstrates the first venture towards fully declarative analysis specification across multiple languages. The key idea is to translate source code directly to Datalog facts in the analysis domain using declarative syntax transformation. We then reuse existing Datalog analyses over generated facts, yielding an end-to-end declarative pipeline. As a first approximation we pursue a syntax-driven approach and demonstrate the feasibility of generating and using lightweight versions of liveness and call graph reachability properties. We then discuss the workability of extending declarative fact generation to also incorporate semantic information.
Comprehensive Movie Recommendation System
Venue: arXiv cs.IR 2021
Year: 2021

A recommender system, also known as a recommendation system, is a type of information filtering system that attempts to forecast a user's rating or preference for an item. This article designs and implements a complete movie recommendation system prototype based on the Genre, Pearson Correlation Coefficient, Cosine Similarity, KNN-Based, Content-Based Filtering using TFIDF and SVD, Collaborative Filtering using TFIDF and SVD, Surprise Library based recommendation system technology. Apart from that in this paper, we present a novel idea that applies machine learning techniques to construct a cluster for the movie based on genres and then observes the inertia value number of clusters were defined. The constraints of the approaches discussed in this work have been described, as well as how one strategy overcomes the disadvantages of another. The whole work has been done on the dataset Movie Lens present at the group lens website which contains 100836 ratings and 3683 tag applications across 9742 movies. These data were created by 610 users between March 29, 1996, and September 24, 2018.
TFW2V: An Enhanced Document Similarity Method for the Morphologically Rich Finnish Language
Venue: arXiv cs.CL 2021
Year: 2021

Measuring the semantic similarity of different texts has many important applications in Digital Humanities research such as information retrieval, document clustering and text summarization. The performance of different methods depends on the length of the text, the domain and the language. This study focuses on experimenting with some of the current approaches to Finnish, which is a morphologically rich language. At the same time, we propose a simple method, TFW2V, which shows high efficiency in handling both long text documents and limited amounts of data. Furthermore, we design an objective evaluation method which can be used as a framework for benchmarking text similarity approaches.
LaTr: Layout-Aware Transformer for Scene-Text VQA
Venue: arXiv cs.CV 2021
Year: 2021

We propose a novel multimodal architecture for Scene Text Visual Question Answering (STVQA), named Layout-Aware Transformer (LaTr). The task of STVQA requires models to reason over different modalities. Thus, we first investigate the impact of each modality, and reveal the importance of the language module, especially when enriched with layout information. Accounting for this, we propose a single objective pre-training scheme that requires only text and spatial cues. We show that applying this pre-training scheme on scanned documents has certain advantages over using natural images, despite the domain gap. Scanned documents are easy to procure, text-dense and have a variety of layouts, helping the model learn various spatial cues (e.g. left-of, below etc.) by tying together language and layout information. Compared to existing approaches, our method performs vocabulary-free decoding and, as shown, generalizes well beyond the training vocabulary. We further demonstrate that LaTr improves robustness towards OCR errors, a common reason for failure cases in STVQA. In addition, by leveraging a vision transformer, we eliminate the need for an external object detector. LaTr outperforms state-of-the-art STVQA methods on multiple datasets. In particular, +7.6% on TextVQA, +10.8% on ST-VQA and +4.0% on OCR-VQA (all absolute accuracy numbers).
Emulation of greenhouse-gas sensitivities using variational autoencoders
Venue: arXiv cs.LG 2021
Year: 2021

Flux inversion is the process by which sources and sinks of a gas are identified from observations of gas mole fraction. The inversion often involves running a Lagrangian particle dispersion model (LPDM) to generate sensitivities between observations and fluxes over a spatial domain of interest. The LPDM must be run backward in time for every gas measurement, and this can be computationally prohibitive. To address this problem, here we develop a novel spatio-temporal emulator for LPDM sensitivities that is built using a convolutional variational autoencoder (CVAE). With the encoder segment of the CVAE, we obtain approximate (variational) posterior distributions over latent variables in a low-dimensional space. We then use a spatio-temporal Gaussian process emulator on the low-dimensional space to emulate new variables at prediction locations and time points. Emulated variables are then passed through the decoder segment of the CVAE to yield emulated sensitivities. We show that our CVAE-based emulator outperforms the more traditional emulator built using empirical orthogonal functions and that it can be used with different LPDMs. We conclude that our emulation-based approach can be used to reliably reduce the computing time needed to generate LPDM outputs for use in high-resolution flux inversions.
Collaborative adversary nodes learning on the logs of IoT devices in an IoT network
Venue: arXiv cs.CR 2021
Year: 2021

Artificial Intelligence (AI) development has encouraged many new research areas, including AI-enabled Internet of Things (IoT) network. AI analytics and intelligent paradigms greatly improve learning efficiency and accuracy. Applying these learning paradigms to network scenarios provide technical advantages of new networking solutions. In this paper, we propose an improved approach for IoT security from data perspective. The network traffic of IoT devices can be analyzed using AI techniques. The Adversary Learning (AdLIoTLog) model is proposed using Recurrent Neural Network (RNN) with attention mechanism on sequences of network events in the network traffic. We define network events as a sequence of the time series packets of protocols captured in the log. We have considered different packets TCP packets, UDP packets, and HTTP packets in the network log to make the algorithm robust. The distributed IoT devices can collaborate to cripple our world which is extending to Internet of Intelligence. The time series packets are converted into structured data by removing noise and adding timestamps. The resulting data set is trained by RNN and can detect the node pairs collaborating with each other. We used the BLEU score to evaluate the model performance. Our results show that the predicting performance of the AdLIoTLog model trained by our method degrades by 3-4% in the presence of attack in comparison to the scenario when the network is not under attack. AdLIoTLog can detect adversaries because when adversaries are present the model gets duped by the collaborative events and therefore predicts the next event with a biased event rather than a benign event. We conclude that AI can provision ubiquitous learning for the new generation of Internet of Things.
Preprocessing in Inductive Logic Programming
Venue: arXiv cs.LG 2021
Year: 2021

Inductive logic programming is a type of machine learning in which logic programs are learned from examples. This learning typically occurs relative to some background knowledge provided as a logic program. This dissertation introduces bottom preprocessing, a method for generating initial constraints on the programs an ILP system must consider. Bottom preprocessing applies ideas from inverse entailment to modern ILP systems. Inverse entailment is an influential early ILP approach introduced with Progol. This dissertation also presents $\bot$-Popper, an implementation of bottom preprocessing for the modern ILP system Popper. It is shown experimentally that bottom preprocessing can reduce learning times of ILP systems on hard problems. This reduction can be especially significant when the amount of background knowledge in the problem is large.
RumbleML: program the lakehouse with JSONiq
Venue: arXiv cs.DB 2021
Year: 2021

Lakehouse systems have reached in the past few years unprecedented size and heterogeneity and have been embraced by many industry players. However, they are often difficult to use as they lack the declarative language and optimization possibilities of relational engines. This paper introduces RumbleML, a high-level, declarative library integrated into the RumbleDB engine and with the JSONiq language. RumbleML allows using a single platform for data cleaning, data preparation, training, and inference, as well as management of models and results. It does it using a purely declarative language (JSONiq) for all these tasks and without any performance loss over existing platforms (e.g. Spark). The key insights of the design of RumbleML are that training sets, evaluation sets, and test sets can be represented as homogeneous sequences of flat objects; that models can be seamlessly embodied in function items mapping input test sets into prediction-augmented result sets; and that estimators can be seamlessly embodied in function items mapping input training sets to models. We argue that this makes JSONiq a viable and seamless programming language for data lakehouses across all their features, whether database-related or machine-learning-related. While lakehouses bring Machine Learning and Data Wrangling on the same platform, RumbleML also brings them to the same language, JSONiq. In the paper, we present the first prototype and compare its performance to Spark showing the benefit of a huge functionality and productivity gain for cleaning up, normalizing, validating data, feeding it into Machine Learning pipelines, and analyzing the output, all within the same system and language and at scale.
Towards more patient friendly clinical notes through language models and ontologies
Venue: arXiv cs.CL 2021
Year: 2021

Clinical notes are an efficient way to record patient information but are notoriously hard to decipher for non-experts. Automatically simplifying medical text can empower patients with valuable information about their health, while saving clinicians time. We present a novel approach to automated simplification of medical text based on word frequencies and language modelling, grounded on medical ontologies enriched with layman terms. We release a new dataset of pairs of publicly available medical sentences and a version of them simplified by clinicians. Also, we define a novel text simplification metric and evaluation framework, which we use to conduct a large-scale human evaluation of our method against the state of the art. Our method based on a language model trained on medical forum data generates simpler sentences while preserving both grammar and the original meaning, surpassing the current state of the art.
Dynamic Page Placement on Real Persistent Memory Systems
Venue: arXiv cs.DC 2021
Year: 2021

As persistent memory (PM) technologies emerge, hybrid memory architectures combining DRAM with PM bring the potential to provide a tiered, byte-addressable main memory of unprecedented capacity. Nearly a decade after the first proposals for these hybrid architectures, the real technology has finally reached commercial availability with Intel Optane(TM) DC Persistent Memory (DCPMM). This raises the challenge of designing systems that realize this potential in practice, namely through effective approaches that dynamically decide at which memory tier should pages be placed. In this paper, we are the first, to our knowledge, to systematically analyze tiered page placement on real DCPMM-based systems. To this end, we start by revisiting the assumptions of state-of-the-art proposals, and confronting them with the idiosyncrasies of today's off-the-shelf DCPMM-equipped architectures. This empirical study reveals that some of the key design choices in the literature rely on important assumptions that are not verified in present-day DRAM-DCPMM memory architectures. Based on the lessons from this study, we design and implement HyPlacer, a tool for tiered page placement in off-the-shelf Linux-based systems equipped with DRAM+DCPMM. In contrast to previous proposals, HyPlacer follows an approach guided by two main practicality principles: 1) it is tailored to the performance idiosyncrasies of off-theshelf DRAM+DCPMM systems; and 2) it can be seamlessly integrated into Linux with minimal kernel-mode components, while ensuring extensibility to other HMAs and other data placement policies. Our experimental evaluation of HyPlacer shows that it outperforms both solutions proposed in past literature and placement options that are currently available in off-the-shelf DCPMM-equipped Linux systems, reaching an improvement of up to 11x when compared to the default memory policy in Linux.
Digital Editions as Distant Supervision for Layout Analysis of Printed Books
Venue: arXiv cs.CV 2021
Year: 2021

Archivists, textual scholars, and historians often produce digital editions of historical documents. Using markup schemes such as those of the Text Encoding Initiative and EpiDoc, these digital editions often record documents' semantic regions (such as notes and figures) and physical features (such as page and line breaks) as well as transcribing their textual content. We describe methods for exploiting this semantic markup as distant supervision for training and evaluating layout analysis models. In experiments with several model architectures on the half-million pages of the Deutsches Textarchiv (DTA), we find a high correlation of these region-level evaluation methods with pixel-level and word-level metrics. We discuss the possibilities for improving accuracy with self-training and the ability of models trained on the DTA to generalize to other historical printed books.
Deterministic Parallel Hypergraph Partitioning
Venue: arXiv cs.DS 2021
Year: 2021

Balanced hypergraph partitioning is a classical NP-hard optimization problem with applications in various domains such as VLSI design, simulating quantum circuits, optimizing data placement in distributed databases or minimizing communication volume in high-performance computing. Engineering parallel heuristics for this problem is a topic of recent research. Most of them are non-deterministic though. In this work, we design and implement a highly scalable deterministic algorithm in the state-of-the-art parallel partitioning framework Mt-KaHyPar. On our extensive set of benchmark instances, it achieves similar partition quality and performance as a comparable but non-deterministic configuration of Mt-KaHyPar and outperforms the only other existing parallel deterministic algorithm BiPart regarding partition quality, running time and parallel speedups.