OpenRaiser/Intern-Atlas huggingface dataset:
A Hybrid Deep Learning Model for Arabic Text Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Arabic text recognition is a challenging task because of the cursive nature of Arabic writing system, its joint writing scheme, the large number of ligatures and many other challenges. Deep Learning DL models achieved significant progress in numerous domains including computer vision and sequence modelling. This paper presents a model that can recognize Arabic text that was printed using multiple font types including fonts that mimic Arabic handwritten scripts. The proposed model employs a hybrid DL network that can recognize Arabic printed text without the need for character segmentation. The model was tested on a custom dataset comprised of over two million word samples that were generated using 18 different Arabic font types. The objective of the testing process was to assess the model capability in recognizing a diverse set of Arabic fonts representing a varied cursive styles. The model achieved good results in recognizing characters and words and it also achieved promising results in recognizing characters when it was tested on unseen data. The prepared model, the custom datasets and the toolkit for generating similar datasets are made publicly available, these tools can be used to prepare models for recognizing other font types as well as to further extend and enhance the performance of the proposed model.
Towards Musically Meaningful Explanations Using Source Separation
Venue: arXiv cs.SD 2020
Year: 2020

Deep neural networks (DNNs) are successfully applied in a wide variety of music information retrieval (MIR) tasks. Such models are usually considered "black boxes", meaning that their predictions are not interpretable. Prior work on explainable models in MIR has generally used image processing tools to produce explanations for DNN predictions, but these are not necessarily musically meaningful, or can be listened to (which, arguably, is important in music). We propose audioLIME, a method based on Local Interpretable Model-agnostic Explanation (LIME), extended by a musical definition of locality. LIME learns locally linear models on perturbations of an example that we want to explain. Instead of extracting components of the spectrogram using image segmentation as part of the LIME pipeline, we propose using source separation. The perturbations are created by switching on/off sources which makes our explanations listenable. We first validate audioLIME on a classifier that was deliberately trained to confuse the true target with a spurious signal, and show that this can easily be detected using our method. We then show that it passes a sanity check that many available explanation methods fail. Finally, we demonstrate the general applicability of our (model-agnostic) method on a third-party music tagger.
Augmenting Sheet Music with Rhythmic Fingerprints
Venue: arXiv cs.HC 2020
Year: 2020

In this paper, we bridge the gap between visualization and musicology by focusing on rhythm analysis tasks, which are tedious due to the complex visual encoding of the well-established Common Music Notation (CMN). Instead of replacing the CMN, we augment sheet music with rhythmic fingerprints to mitigate the complexity originating from the simultaneous encoding of musical features. The proposed visual design exploits music theory concepts such as the rhythm tree to facilitate the understanding of rhythmic information. Juxtaposing sheet music and the rhythmic fingerprints maintains the connection to the familiar representation. To investigate the usefulness of the rhythmic fingerprint design for identifying and comparing rhythmic patterns, we conducted a controlled user study with four experts and four novices. The results show that the rhythmic fingerprints enable novice users to recognize rhythmic patterns that only experts can identify using non-augmented sheet music.
ViS-Á-ViS : Detecting Similar Patterns in Annotated Literary Text
Venue: arXiv cs.HC 2020
Year: 2020

We present a web-based system called ViS-Á-ViS aiming to assist literary scholars in detecting repetitive patterns in an annotated textual corpus. Pattern detection is made possible using distant reading visualizations that highlight potentially interesting patterns. In addition, the system uses time-series alignment algorithms, and in particular, dynamic time warping (DTW), to detect patterns automatically. We present a case-study where an ancient Hebrew poetry corpus was manually annotated with figurative language devices as metaphors and similes and then loaded into the system. Preliminary results confirm the effectiveness of the system in analyzing the annotated data and in detecting literary patterns and similarities.
Domain Priori Knowledge based Integrated Solution Design for Internet of Services
Venue: arXiv cs.SE 2020
Year: 2020

Various types of services, such as web APIs, IoT services, O2O services, and many others, have flooded on the Internet. Interconnections among these services have resulted in a new phenomenon called "Internet of Services" (IoS). By IoS,people don't need to request multiple services by themselves to fulfill their daily requirements, but it is an IoS platform that is responsible for constructing integrated solutions for them. Since user requirements (URs) are usually coarse-grained and transboundary, IoS platforms have to integrate services from multiple domains to fulfill the requirements. Considering there are too many available services in IoS, a big challenge is how to look for a tradeoff between the construction efficiency and the precision of final solutions. For this challenge, we introduce a framework and a platform for transboundary user requirement oriented solution design in IoS. The main idea is to make use of domain priori knowledge derived from the commonness and similarities among massive historical URs and among historical integrated service solutions(ISSs). Priori knowledge is classified into three types: requirement patterns (RPs), service patterns (SPs), and probabilistic matching matrix (PMM) between RPs and SPs. A UR is modeled in the form of an intention tree (ITree) along with a set of constraints on intention nodes, and then optimal RPs are selected to cover the I-Tree as much as possible. By taking advantage of the PMM, a set of SPs are filtered out and composed together to form the final ISS. Finally, the design of a platform supporting the above process is introduced.
HyperTendril: Visual Analytics for User-Driven Hyperparameter Optimization of Deep Neural Networks
Venue: arXiv cs.HC 2020
Year: 2020

To mitigate the pain of manually tuning hyperparameters of deep neural networks, automated machine learning (AutoML) methods have been developed to search for an optimal set of hyperparameters in large combinatorial search spaces. However, the search results of AutoML methods significantly depend on initial configurations, making it a non-trivial task to find a proper configuration. Therefore, human intervention via a visual analytic approach bears huge potential in this task. In response, we propose HyperTendril, a web-based visual analytics system that supports user-driven hyperparameter tuning processes in a model-agnostic environment. HyperTendril takes a novel approach to effectively steering hyperparameter optimization through an iterative, interactive tuning procedure that allows users to refine the search spaces and the configuration of the AutoML method based on their own insights from given results. Using HyperTendril, users can obtain insights into the complex behaviors of various hyperparameter search algorithms and diagnose their configurations. In addition, HyperTendril supports variable importance analysis to help the users refine their search spaces based on the analysis of relative importance of different hyperparameters and their interaction effects. We present the evaluation demonstrating how HyperTendril helps users steer their tuning processes via a longitudinal user study based on the analysis of interaction logs and in-depth interviews while we deploy our system in a professional industrial environment.
Going Beyond T-SNE: Exposing \texttt{whatlies} in Text Embeddings
Venue: arXiv cs.CL 2020
Year: 2020

We introduce whatlies, an open source toolkit for visually inspecting word and sentence embeddings. The project offers a unified and extensible API with current support for a range of popular embedding backends including spaCy, tfhub, huggingface transformers, gensim, fastText and BytePair embeddings. The package combines a domain specific language for vector arithmetic with visualisation tools that make exploring word embeddings more intuitive and concise. It offers support for many popular dimensionality reduction techniques as well as many interactive visualisations that can either be statically exported or shared via Jupyter notebooks. The project documentation is available from https://rasahq.github.io/whatlies/.
Homomorphic-Encrypted Volume Rendering
Venue: arXiv cs.CR 2020
Year: 2020

Computationally demanding tasks are typically calculated in dedicated data centers, and real-time visualizations also follow this trend. Some rendering tasks, however, require the highest level of confidentiality so that no other party, besides the owner, can read or see the sensitive data. Here we present a direct volume rendering approach that performs volume rendering directly on encrypted volume data by using the homomorphic Paillier encryption algorithm. This approach ensures that the volume data and rendered image are uninterpretable to the rendering server. Our volume rendering pipeline introduces novel approaches for encrypted-data compositing, interpolation, and opacity modulation, as well as simple transfer function design, where each of these routines maintains the highest level of privacy. We present performance and memory overhead analysis that is associated with our privacy-preserving scheme. Our approach is open and secure by design, as opposed to secure through obscurity. Owners of the data only have to keep their secure key confidential to guarantee the privacy of their volume data and the rendered images. Our work is, to our knowledge, the first privacy-preserving remote volume-rendering approach that does not require that any server involved be trustworthy; even in cases when the server is compromised, no sensitive data will be leaked to a foreign party.
Towards A Learning-Based Framework for Self-Driving Design of Networking Protocols
Venue: arXiv cs.NI 2020
Year: 2020

Networking protocols are designed through long-time and hard-work human efforts. Machine Learning (ML)-based solutions have been developed for communication protocol design to avoid manual efforts to tune individual protocol parameters. While other proposed ML-based methods mainly focus on tuning individual protocol parameters (e.g., adjusting contention window), our main contribution is to propose a novel Deep Reinforcement Learning (DRL)-based framework to systematically design and evaluate networking protocols. We decouple a protocol into a set of parametric modules, each representing a main protocol functionality that is used as DRL input to better understand the generated protocols design optimization and analyze them in a systematic fashion. As a case study, we introduce and evaluate DeepMAC a framework in which a MAC protocol is decoupled into a set of blocks across popular flavors of 802.11 WLANs (e.g., 802.11 b/a/g/n/ac). We are interested to see what blocks are selected by DeepMAC across different networking scenarios and whether DeepMAC is able to adapt to network dynamics.
5G Technologies Based Remote E-Health: Architecture, Applications, and Solutions
Venue: arXiv cs.NI 2020
Year: 2020

Currently, many countries are facing the problems of aging population, serious imbalance of medical resources supply and demand, as well as uneven geographical distribution, resulting in a huge demand for remote e-health. Particularly, with invasions of COVID-19, the health of people and even social stability have been challenged unprecedentedly. To contribute to these urgent problems, this article proposes a general architecture of the remote e-health, where the city hospital provides the technical supports and services for remote hospitals. Meanwhile, 5G technologies supported telemedicine is introduced to satisfy the high-speed transmission of massive multimedia medical data, and further realize the sharing of medical resources. Moreover, to turn passivity into initiative to prevent COVID-19, a broad area epidemic prevention and control scheme is also investigated, especially for the remote areas. We discuss their principles and key features, and foresee the challenges, opportunities, and future research trends. Finally, a node value and content popularity based caching strategy is introduced to provide a preliminary solution of the massive data storage and low-latency transmission.
Naive Artificial Intelligence
Venue: arXiv cs.AI 2020
Year: 2020

In the cognitive sciences, it is common to distinguish between crystal intelligence, the ability to utilize knowledge acquired through past learning or experience and fluid intelligence, the ability to solve novel problems without relying on prior knowledge. Using this cognitive distinction between the two types of intelligence, extensively-trained deep networks that can play chess or Go exhibit crystal but not fluid intelligence. In humans, fluid intelligence is typically studied and quantified using intelligence tests. Previous studies have shown that deep networks can solve some forms of intelligence tests, but only after extensive training. Here we present a computational model that solves intelligence tests without any prior training. This ability is based on continual inductive reasoning, and is implemented by deep unsupervised latent-prediction networks. Our work demonstrates the potential fluid intelligence of deep networks. Finally, we propose that the computational principles underlying our approach can be used to model fluid intelligence in the cognitive sciences.
Interactive Visual Study of Multiple Attributes Learning Model of X-Ray Scattering Images
Venue: arXiv cs.CV 2020
Year: 2020

Existing interactive visualization tools for deep learning are mostly applied to the training, debugging, and refinement of neural network models working on natural images. However, visual analytics tools are lacking for the specific application of x-ray image classification with multiple structural attributes. In this paper, we present an interactive system for domain scientists to visually study the multiple attributes learning models applied to x-ray scattering images. It allows domain scientists to interactively explore this important type of scientific images in embedded spaces that are defined on the model prediction output, the actual labels, and the discovered feature space of neural networks. Users are allowed to flexibly select instance images, their clusters, and compare them regarding the specified visual representation of attributes. The exploration is guided by the manifestation of model performance related to mutual relationships among attributes, which often affect the learning accuracy and effectiveness. The system thus supports domain scientists to improve the training dataset and model, find questionable attributes labels, and identify outlier images or spurious data clusters. Case studies and scientists feedback demonstrate its functionalities and usefulness.
Externalizing Transformations of Historical Documents: Opportunities for Provenance-Driven Visualization
Venue: arXiv cs.HC 2020
Year: 2020

Transcription, annotation, digitization and/or visualization are common transformations that historical documents such as national records, birth/death registers, university records, letters or books undergo. Reasons for those transformations span from the (physical) protection of the original materials to disclosure of 'hidden' information or patterns within the documents. Even though such transformations bring new insights and perspectives on the documents, they also modify the documents' content, structure, and/or artifactual form and thus, occlude prior knowledge and interpretation. When it comes to visualization as a means to transform historical documents from written to abstract visual form, there is typically little acknowledgment or even understanding of the previous transformation steps these documents have gone through. The 'tremendous rhetorical force' of visualization, we argue, should not be at the expense of the multiple pasts, contexts, and curators that are inherent in historical record collections. Rather, the urgent question for the fields of visualization and the (digital) humanities is how to better support awareness of these multiple layers of interpretation and the people behind them when representing historical documents. We begin to address this question based on a collection of historical university records by (a) investigating common transformation processes of historical documents, and (b) discussing opportunities and challenges for making such transformations transparent through what we call 'provenance-driven visualization'; the idea for a visualization that makes visible the layers of transformation (including interpretation, re-structuring, and curation) inherent in historical documents.
Chordal Decomposition for Spectral Coarsening
Venue: arXiv cs.GR 2020
Year: 2020

We introduce a novel solver to significantly reduce the size of a geometric operator while preserving its spectral properties at the lowest frequencies. We use chordal decomposition to formulate a convex optimization problem which allows the user to control the operator sparsity pattern. This allows for a trade-off between the spectral accuracy of the operator and the cost of its application. We efficiently minimize the energy with a change of variables and achieve state-of-the-art results on spectral coarsening. Our solver further enables novel applications including volume-to-surface approximation and detaching the operator from the mesh, i.e., one can produce a mesh tailormade for visualization and optimize an operator separately for computation.
End-to-End Deep Learning Model for Cardiac Cycle Synchronization from Multi-View Angiographic Sequences
Venue: arXiv cs.CV 2020
Year: 2020

Dynamic reconstructions (3D+T) of coronary arteries could give important perfusion details to clinicians. Temporal matching of the different views, which may not be acquired simultaneously, is a prerequisite for an accurate stereo-matching of the coronary segments. In this paper, we show how a neural network can be trained from angiographic sequences to synchronize different views during the cardiac cycle using raw x-ray angiography videos exclusively. First, we train a neural network model with angiographic sequences to extract features describing the progression of the cardiac cycle. Then, we compute the distance between the feature vectors of every frame from the first view with those from the second view to generate distance maps that display stripe patterns. Using pathfinding, we extract the best temporally coherent associations between each frame of both videos. Finally, we compare the synchronized frames of an evaluation set with the ECG signals to show an alignment with 96.04% accuracy.
Literal Encoding: Text is a first-class data encoding
Venue: arXiv cs.HC 2020
Year: 2020

Digital humanities are rooted in text analysis. However, most visualization paradigms use only categoric, ordered or quantitative data. Literal text must be considered a base data type to encode into visualizations. Literal text offers functional, perceptual, cognitive, semantic and operational benefits. These are briefly illustrated with a subset of sample visualizations focused on semantic word sequences, indicating benefits over standard graphs, maps, treemaps, bar charts and narrative layouts.
Supporting Expert Close Analysis of Historical Scientific Writings: A Case Study for Near-by Reading
Venue: arXiv cs.HC 2020
Year: 2020

Distant reading methodologies make use of computational processes to aid in the analysis of large text corpora which might not be pliable to traditional methods of scholarly analysis due to their volume. While these methods have been applied effectively to a variety of types of texts and contexts, they can leave unaddressed the needs of scholars in the humanities disciplines like history, who often engage in close reading of sources. Complementing the close analysis of texts with some of the tools of distant reading, such as visualization, can resolve some of the issues. We focus on a particular category of this intersection---which we refer to as near-by reading---wherein an expert engages in a computer-mediated analysis of a text with which they are familiar. We provide an example of this approach by developing a visual analysis application for the near-by reading of 19th-century scientific writings by J. W. von Goethe and A. P. de Candolle. We show that even the most formal and public texts, such as scientific treatises, can reveal unexpressed personal biases and philosophies that the authors themselves might not have recognized.
BP-RRT: Barrier Pair Synthesis for Temporal Logic Motion Planning
Venue: arXiv cs.RO 2020
Year: 2020

For a nonlinear system (e.g. a robot) with its continuous state space trajectories constrained by a linear temporal logic specification, the synthesis of a low-level controller for mission execution often results in a non-convex optimization problem. We devise a new algorithm to solve this type of non-convex problems by formulating a rapidly-exploring random tree of barrier pairs, with each barrier pair composed of a quadratic barrier function and a full state feedback controller. The proposed method employs a rapid-exploring random tree to deal with the non-convex constraints and uses barrier pairs to fulfill the local convex constraints. As such, the method solves control problems fulfilling the required transitions of an automaton in order to satisfy given linear temporal logic constraints. At the same time it synthesizes locally optimal controllers in order to transition between the regions corresponding to the alphabet of the automaton. We demonstrate this new algorithm on a simulation of a two linkage manipulator robot.
A Large Scale Empirical Study of the Impact of Spaghetti Code and Blob Anti-patterns on Program Comprehension
Venue: arXiv cs.SE 2020
Year: 2020

Context: Several studies investigated the impact of anti-patterns (i.e., "poor" solutions to recurring design problems) during maintenance activities and reported that anti-patterns significantly affect the developers' effort required to edit files. However, before developers edit files, they must understand the source code of the systems. This source code must be easy to understand by developers. Objective: In this work, we provide a complete assessment of the impact of two instances of two anti-patterns, Blob or Spaghetti Code, on program comprehension. Method: We analyze the impact of these two anti-patterns through three empirical studies conducted at Polytechnique Montréal (Canada) with 24 participants; at Carlton University (Canada) with 30 participants; and at University Basilicata (Italy) with 79 participants. Results: We collect data from 372 tasks obtained thanks to 133 different participants from the three universities. We use three metrics to assess the developers' comprehension of the source code: (1) the duration to complete each task; (2) their percentage of correct answers; and, (3) the NASA task load index for their effort. Conclusions: We report that, although single occurrences of Blob or Spaghetti code anti-patterns have little effect on code comprehension, two occurrences of either Blob or Spaghetti Code significantly increases the developers' time spent in their tasks, reduce their percentage of correct answers, and increase their effort. Hence, we recommend that developers act on both anti-patterns, which should be refactored out of the source code whenever possible. We also recommend further studies on combinations of anti-patterns rather than on single anti-patterns one at a time.