OpenRaiser/Intern-Atlas huggingface dataset:
Modeling and Understanding Ethereum Transaction Records via a Complex Network Approach
Venue: arXiv cs.SI 2020
Year: 2020

As the largest public blockchain-based platform supporting smart contracts, Ethereum has accumulated a large number of user transaction records since its debut in 2014. Analysis of Ethereum transaction records, however, is still relatively unexplored till now. Modeling the transaction records as a static simple graph, existing methods are unable to accurately characterize the temporal and multiplex features of the edges. In this brief, we first model the Ethereum transaction records as a complex network by incorporating time and amount features of the transactions, and then design several flexible temporal walk strategies for random-walk based graph representation of this large-scale network. Experiments of temporal link prediction on real Ethereum data demonstrate that temporal information and multiplicity characteristic of edges are indispensable for accurate modeling and understanding of Ethereum transaction networks.
Towards Zero-Shot Knowledge Distillation for Natural Language Processing
Venue: arXiv cs.CL 2020
Year: 2020

Knowledge Distillation (KD) is a common knowledge transfer algorithm used for model compression across a variety of deep learning based natural language processing (NLP) solutions. In its regular manifestations, KD requires access to the teacher's training data for knowledge transfer to the student network. However, privacy concerns, data regulations and proprietary reasons may prevent access to such data. We present, to the best of our knowledge, the first work on Zero-Shot Knowledge Distillation for NLP, where the student learns from the much larger teacher without any task specific data. Our solution combines out of domain data and adversarial training to learn the teacher's output distribution. We investigate six tasks from the GLUE benchmark and demonstrate that we can achieve between 75% and 92% of the teacher's classification score (accuracy or F1) while compressing the model 30 times.
Flexible model composition in machine learning and its implementation in MLJ
Venue: arXiv cs.LG 2020
Year: 2020

A graph-based protocol called `learning networks' which combine assorted machine learning models into meta-models is described. Learning networks are shown to overcome several limitations of model composition as implemented in the dominant machine learning platforms. After illustrating the protocol in simple examples, a concise syntax for specifying a learning network, implemented in the MLJ framework, is presented. Using the syntax, it is shown that learning networks are are sufficiently flexible to include Wolpert's model stacking, with out-of-sample predictions for the base learners.
Optimal Software Architecture From Initial Requirements: An End-to-End Approach
Venue: arXiv cs.SE 2020
Year: 2020

A software architect turns system requirements into a suitable software architecture through an architecture optimization process. However, how should the architect decide which quality improvement to prioritize, e.g., security or reliability? In software product line, should a small improvement in multiple products be preferred over a large improvement in a single product? Existing architecture optimization methods handle various steps in the process, but none of them systematically guides the architect in generating an optimal architecture from the initial requirements. In this work we present an end-to-end approach for generating an optimal software architecture for a single software product and an optimal family of architectures for a family of products. We report on a case-study of applying our approach to optimize five industry-grade products in a real-life product line architecture, where 359 possible combinations of ten different quality efforts were prioritized.
HopRetriever: Retrieve Hops over Wikipedia to Answer Complex Questions
Venue: arXiv cs.CL 2020
Year: 2020

Collecting supporting evidence from large corpora of text (e.g., Wikipedia) is of great challenge for open-domain Question Answering (QA). Especially, for multi-hop open-domain QA, scattered evidence pieces are required to be gathered together to support the answer extraction. In this paper, we propose a new retrieval target, hop, to collect the hidden reasoning evidence from Wikipedia for complex question answering. Specifically, the hop in this paper is defined as the combination of a hyperlink and the corresponding outbound link document. The hyperlink is encoded as the mention embedding which models the structured knowledge of how the outbound link entity is mentioned in the textual context, and the corresponding outbound link document is encoded as the document embedding representing the unstructured knowledge within it. Accordingly, we build HopRetriever which retrieves hops over Wikipedia to answer complex questions. Experiments on the HotpotQA dataset demonstrate that HopRetriever outperforms previously published evidence retrieval methods by large margins. Moreover, our approach also yields quantifiable interpretations of the evidence collection process.
Deep Graph Generators: A Survey
Venue: arXiv cs.LG 2020
Year: 2020

Deep generative models have achieved great success in areas such as image, speech, and natural language processing in the past few years. Thanks to the advances in graph-based deep learning, and in particular graph representation learning, deep graph generation methods have recently emerged with new applications ranging from discovering novel molecular structures to modeling social networks. This paper conducts a comprehensive survey on deep learning-based graph generation approaches and classifies them into five broad categories, namely, autoregressive, autoencoder-based, RL-based, adversarial, and flow-based graph generators, providing the readers a detailed description of the methods in each class. We also present publicly available source codes, commonly used datasets, and the most widely utilized evaluation metrics. Finally, we highlight the existing challenges and discuss future research directions.
Matching on the line admits no $o(\sqrt{\log n})$-competitive algorithm
Venue: arXiv cs.DS 2020
Year: 2020

We present a simple proof that the competitive ratio of any randomized online matching algorithm for the line is at least $\sqrt{\log_2(n\!+\!1)}/12$ for all $n=2^i\!-\!1: i\in\mathbb{N}$.
Improving Unsupervised Domain Adaptation by Reducing Bi-level Feature Redundancy
Venue: arXiv cs.LG 2020
Year: 2020

Reducing feature redundancy has shown beneficial effects for improving the accuracy of deep learning models, thus it is also indispensable for the models of unsupervised domain adaptation (UDA). Nevertheless, most recent efforts in the field of UDA ignores this point. Moreover, main schemes realizing this in general independent of UDA purely involve a single domain, thus might not be effective for cross-domain tasks. In this paper, we emphasize the significance of reducing feature redundancy for improving UDA in a bi-level way. For the first level, we try to ensure compact domain-specific features with a transferable decorrelated normalization module, which preserves specific domain information whilst easing the side effect of feature redundancy on the sequel domain-invariance. In the second level, domain-invariant feature redundancy caused by domain-shared representation is further mitigated via an alternative brand orthogonality for better generalization. These two novel aspects can be easily plugged into any BN-based backbone neural networks. Specifically, simply applying them to ResNet50 has achieved competitive performance to the state-of-the-arts on five popular benchmarks. Our code will be available at https://github.com/dreamkily/gUDA.
On the Construction of a Post-Quantum Blockchain for Smart City
Venue: arXiv cs.CR 2020
Year: 2020

Owing to some special characteristics and features, blockchain is a very useful technique that can securely organize diverse devices in a smart city. It finds wide applications, especially in distributed environments, where entities such as wireless sensors need to be certain of the authenticity of the server. As contemporary blockchain techniques that address post-quantum concerns have not been designed, in this study, we investigate a blockchain in the post-quantum setting and seek to discover how it can resist attacks from quantum computing. In addition, traditional proof of work (PoW)-based consensus protocols such as Bitcoin cannot supply memory mining, and the transaction capacity of each block in a blockchain is limited and needs to be expanded. Thus, a new post-quantum proof of work (post-quantum PoW) consensus algorithm for security and privacy of smart city applications is proposed. It can be used to not only protect a blockchain under a quantum computing attack compared to existing classical hash-based PoW algorithms but also to supply memory mining. Meanwhile, an identity-based post-quantum signature is embedded into a transaction process to construct lightweight transactions. Subsequently, we provide a detailed description on the execution of the post-quantum lightweight transaction in a blockchain. Overall, this work can help enrich the research on future post-quantum blockchain and support the construction or architecture of emerging blockchain-based smart cities.
UCCA's Foundational Layer: Annotation Guidelines v2.1
Venue: arXiv cs.CL 2020
Year: 2020

This is the annotation manual for Universal Conceptual Cognitive Annotation (UCCA; Abend and Rappoport, 2013), specifically the Foundational Layer. UCCA is a graph-based semantic annotation scheme based on typological linguistic principles. It has been applied to several languages; for ease of exposition these guidelines give examples mainly in English. New annotators may wish to start with the tutorial on the UCCA framework (Abend et al., 2020). Further resources are available at the project homepage: https://universalconceptualcognitiveannotation.github.io
Using Natural Language Relations between Answer Choices for Machine Comprehension
Venue: arXiv cs.CL 2020
Year: 2020

When evaluating an answer choice for Reading Comprehension task, other answer choices available for the question and the answers of related questions about the same paragraph often provide valuable information. In this paper, we propose a method to leverage the natural language relations between the answer choices, such as entailment and contradiction, to improve the performance of machine comprehension. We use a stand-alone question answering (QA) system to perform QA task and a Natural Language Inference (NLI) system to identify the relations between the choice pairs. Then we perform inference using an Integer Linear Programming (ILP)-based relational framework to re-evaluate the decisions made by the standalone QA system in light of the relations identified by the NLI system. We also propose a multitask learning model that learns both the tasks jointly.
Random Embeddings with Optimal Accuracy
Venue: arXiv cs.LG 2020
Year: 2020

This work constructs Jonson-Lindenstrauss embeddings with best accuracy, as measured by variance, mean-squared error and exponential concentration of the length distortion. Lower bounds for any data and embedding dimensions are determined, and accompanied by matching and efficiently samplable constructions (built on orthogonal matrices). Novel techniques: a unit sphere parametrization, the use of singular-value latent variables and Schur-convexity are of independent interest.
A Multi-modal Deep Learning Model for Video Thumbnail Selection
Venue: arXiv cs.CV 2020
Year: 2020

Thumbnail is the face of online videos. The explosive growth of videos both in number and variety underpins the importance of a good thumbnail because it saves potential viewers time to choose videos and even entice them to click on them. A good thumbnail should be a frame that best represents the content of a video while at the same time capturing viewers' attention. However, the techniques and models in the past only focus on frames within a video, and we believe such narrowed focus leave out much useful information that are part of a video. In this paper, we expand the definition of content to include title, description, and audio of a video and utilize information provided by these modalities in our selection model. Specifically, our model will first sample frames uniformly in time and return the top 1,000 frames in this subset with the highest aesthetic scores by a Double-column Convolutional Neural Network, to avoid the computational burden of processing all frames in downstream task. Then, the model incorporates frame features extracted from VGG16, text features from ELECTRA, and audio features from TRILL. These models were selected because of their results on popular datasets as well as their competitive performances. After feature extraction, the time-series features, frames and audio, will be fed into Transformer encoder layers to return a vector representing their corresponding modality. Each of the four features (frames, title, description, audios) will pass through a context gating layer before concatenation. Finally, our model will generate a vector in the latent space and select the frame that is most similar to this vector in the latent space. To the best of our knowledge, we are the first to propose a multi-modal deep learning model to select video thumbnail, which beats the result from the previous State-of-The-Art models.
Bosonic Random Walk Networks for Graph Learning
Venue: arXiv cs.LG 2020
Year: 2020

The development of Graph Neural Networks (GNNs) has led to great progress in machine learning on graph-structured data. These networks operate via diffusing information across the graph nodes while capturing the structure of the graph. Recently there has also seen tremendous progress in quantum computing techniques. In this work, we explore applications of multi-particle quantum walks on diffusing information across graphs. Our model is based on learning the operators that govern the dynamics of quantum random walkers on graphs. We demonstrate the effectiveness of our method on classification and regression tasks.
PHP code smells in web apps: survival and anomalies
Venue: arXiv cs.SE 2020
Year: 2020

Context: Code smells are considered symptoms of poor design, leading to future problems, such as reduced maintainability. Except for anecdotal cases (e. g. code dropout), a code smell survives until it gets explicitly refactored or removed. This paper presents a longitudinal study on the survival of code smells for web apps built with PHP. Objectives: RQ: (i) code smells survival depends on their scope? (ii) practitioners attitudes towards code smells removal in web apps have changed throughout time? (iii) how long code smells survive in web applications? (iv) are there sudden variations (anomalies) in the density of code smells through the evolution of web apps? Method: We analyze the evolution of 6 code smells in 8 web applications written in PHP at the server side, across several years, using the survival analysis technique. We classify code smells according to scope in two categories: scattered and localized. Scattered code smells are expected to be more harmful since their influence is not circumscribed as in localized code smells. We split the observations for each web app into two equal and consecutive timeframes, to test the hypothesis that code smells awareness has increased throughout time. As for the anomalies, we standardize their detection criteria. Results: We present some evidence that code smells survival depends on their scope: the average survival rate decreases in some of them, while the opposite is observed for the remainder. The survival of localized code smells is around 4 years, while the scattered ones live around 5 years. Around 60% of the smells are removed, and some live through all the application life. We also show how a graphical representation of anomalies found in the evolution of code smells allows unveiling the story of a development project and make managers aware of the need for enforcing regular refactoring practices.
Analysis of Filter Size Effect In Deep Learning
Venue: arXiv cs.CV 2020
Year: 2020

With the use of deep learning in many areas, how to improve this technology or how to develop the structure used more effectively and in a shorter time is an issue that is of interest to many people working in this field. Many studies are carried out on this subject, it is aimed to reduce the duration of the operation and the processing power required, except to obtain the best result with the changes made in the variables, functions and data in the models used. In this study, in the leaf classification made using Mendeley data set consisting of leaf images with a fixed background, all other variables such as layer number, iteration, number of layers in the model and pooling process were kept constant, except for the filter dimensions of the convolution layers in the determined model. Convolution layers in 3 different filter sizes and in addition to this, many results obtained in 2 different structures, increasing and decreasing, and 3 different image sizes were examined. In the literature, it is seen that different uses of pooling layers, changes due to increase or decrease in the number of layers, the difference in the size of the data used, and the results of many functions used with different parameters are evaluated. In the leaf classification of the determined data set with CNN, the change in the filter size of the convolution layer together with the change in different filter combinations and in different sized images was focused. Using the data set and data reproduction methods, it was aimed to make the differences in filter sizes and image sizes more distinct. Using the fixed number of iterations, model and data set, the effect of different filter sizes has been observed.
Boarding House Renting Price Prediction Using Deep Neural Network Regression on Mobile Apps
Venue: arXiv cs.CY 2020
Year: 2020

Boarding house is the most important requirement, especially for college students who live far away from the city, place of his origin or house. However, the problem we see now is the uneven distribution of study places in Indonesia which 75% of the best top educational institutions come from the island of Java. So, students who are looking for boarding houses rent requires more effort in comparing the various aspects desired. They need to survey one by one to the boarding house they want, even though they can survey online, it still requires more effort to pay attention to the desired facilities one by one. Therefore, we then created an Mobile Application that can predict prices based on student needs by comparing several variables, namely city, area, type of boarding house, and facilities. So, students can easily estimate the ideal price. The results of this study prove that we have succeeded in predicting prices for boarding houses rent well based on the variables we have determined, and modeling that variables using Deep Neural Network Regression.
Foresight AND Hindsight
Venue: arXiv cs.OH 2020
Year: 2020

To model is to represent. The threshold of decidability defines two epistemological choices: one model (or a finite number of models) suffices for representing the dynamics below the undecidable; above this threshold (defined as G-complexity), every model is partial, no complete modeling is possible.
An Efficient K-means Clustering Algorithm for Analysing COVID-19
Venue: arXiv cs.LG 2020
Year: 2020

COVID-19 hits the world like a storm by arising pandemic situations for most of the countries around the world. The whole world is trying to overcome this pandemic situation. A better health care quality may help a country to tackle the pandemic. Making clusters of countries with similar types of health care quality provides an insight into the quality of health care in different countries. In the area of machine learning and data science, the K-means clustering algorithm is typically used to create clusters based on similarity. In this paper, we propose an efficient K-means clustering method that determines the initial centroids of the clusters efficiently. Based on this proposed method, we have determined health care quality clusters of countries utilizing the COVID-19 datasets. Experimental results show that our proposed method reduces the number of iterations and execution time to analyze COVID-19 while comparing with the traditional k-means clustering algorithm.
The Multimodal Driver Monitoring Database: A Naturalistic Corpus to Study Driver Attention
Venue: arXiv cs.CV 2020
Year: 2020

A smart vehicle should be able to monitor the actions and behaviors of the human driver to provide critical warnings or intervene when necessary. Recent advancements in deep learning and computer vision have shown great promise in monitoring human behaviors and activities. While these algorithms work well in a controlled environment, naturalistic driving conditions add new challenges such as illumination variations, occlusions and extreme head poses. A vast amount of in-domain data is required to train models that provide high performance in predicting driving related tasks to effectively monitor driver actions and behaviors. Toward building the required infrastructure, this paper presents the multimodal driver monitoring (MDM) dataset, which was collected with 59 subjects that were recorded performing various tasks. We use the Fi- Cap device that continuously tracks the head movement of the driver using fiducial markers, providing frame-based annotations to train head pose algorithms in naturalistic driving conditions. We ask the driver to look at predetermined gaze locations to obtain accurate correlation between the driver's facial image and visual attention. We also collect data when the driver performs common secondary activities such as navigation using a smart phone and operating the in-car infotainment system. All of the driver's activities are recorded with high definition RGB cameras and time-of-flight depth camera. We also record the controller area network-bus (CAN-Bus), extracting important information. These high quality recordings serve as the ideal resource to train various efficient algorithms for monitoring the driver, providing further advancements in the field of in-vehicle safety systems.
This Face Does Not Exist ... But It Might Be Yours! Identity Leakage in Generative Models
Venue: arXiv cs.CV 2020
Year: 2020

Generative adversarial networks (GANs) are able to generate high resolution photo-realistic images of objects that "do not exist." These synthetic images are rather difficult to detect as fake. However, the manner in which these generative models are trained hints at a potential for information leakage from the supplied training data, especially in the context of synthetic faces. This paper presents experiments suggesting that identity information in face images can flow from the training corpus into synthetic samples without any adversarial actions when building or using the existing model. This raises privacy-related questions, but also stimulates discussions of (a) the face manifold's characteristics in the feature space and (b) how to create generative models that do not inadvertently reveal identity information of real subjects whose images were used for training. We used five different face matchers (face_recognition, FaceNet, ArcFace, SphereFace and Neurotechnology MegaMatcher) and the StyleGAN2 synthesis model, and show that this identity leakage does exist for some, but not all methods. So, can we say that these synthetically generated faces truly do not exist? Databases of real and synthetically generated faces are made available with this paper to allow full replicability of the results discussed in this work.