OpenRaiser/Intern-Atlas huggingface dataset:
A metric for evaluating 3D reconstruction and mapping performance with no ground truthing
Venue: arXiv cs.CV 2021
Year: 2021

It is not easy when evaluating 3D mapping performance because existing metrics require ground truth data that can only be collected with special instruments. In this paper, we propose a metric, dense map posterior (DMP), for this evaluation. It can work without any ground truth data. Instead, it calculates a comparable value, reflecting a map posterior probability, from dense point cloud observations. In our experiments, the proposed DMP is benchmarked against ground truth-based metrics. Results show that DMP can provide a similar evaluation capability. The proposed metric makes evaluating different methods more flexible and opens many new possibilities, such as self-supervised methods and more available datasets.
How do some Bayesian Network machine learned graphs compare to causal knowledge?
Venue: arXiv cs.AI 2021
Year: 2021

The graph of a Bayesian Network (BN) can be machine learned, determined by causal knowledge, or a combination of both. In disciplines like bioinformatics, applying BN structure learning algorithms can reveal new insights that would otherwise remain unknown. However, these algorithms are less effective when the input data are limited in terms of sample size, which is often the case when working with real data. This paper focuses on purely machine learned and purely knowledge-based BNs and investigates their differences in terms of graphical structure and how well the implied statistical models explain the data. The tests are based on four previous case studies whose BN structure was determined by domain knowledge. Using various metrics, we compare the knowledge-based graphs to the machine learned graphs generated from various algorithms implemented in TETRAD spanning all three classes of learning. The results show that, while the algorithms produce graphs with much higher model selection score, the knowledge-based graphs are more accurate predictors of variables of interest. Maximising score fitting is ineffective in the presence of limited sample size because the fitting becomes increasingly distorted with limited data, guiding algorithms towards graphical patterns that share higher fitting scores and yet deviate considerably from the true graph. This highlights the value of causal knowledge in these cases, as well as the need for more appropriate fitting scores suitable for limited data. Lastly, the experiments also provide new evidence that support the notion that results from simulated data tell us little about actual real-world performance.
Personal Data Access Control Through Distributed Authorization
Venue: arXiv cs.CR 2021
Year: 2021

This paper presents an architecture of a Personal Information Management System, in which individuals can define the access to their personal data by means of smart contracts. These smart contracts, running on the Ethereum blockchain, implement access control lists and grant immutability, traceability and verifiability of the references to personal data, which is stored itself in a (possibly distributed) file system. A distributed authorization mechanism is devised, where trust from multiple network nodes is necessary to grant the access to the data. To this aim, two possible alternatives are described: a Secret Sharing scheme and Threshold Proxy Re-Encryption scheme. The performance of these alternatives is experimentally compared in terms of execution time. Threshold Proxy Re-Encryption appears to be faster in different scenarios, in particular when increasing message size, number of nodes and the threshold value, i.e. number of nodes needed to grant the data disclosure.
DNC-Aided SCL-Flip Decoding of Polar Codes
Venue: arXiv cs.IT 2021
Year: 2021

Successive-cancellation list (SCL) decoding of polar codes has been adopted for 5G. However, the performance is not very satisfactory with moderate code length. Heuristic or deep-learning-aided (DL-aided) flip algorithms have been developed to tackle this problem. The key for successful flip decoding is to accurately identify error bit positions. In this work, we propose a new flip algorithm with help of differentiable neural computer (DNC). New state and action encoding are developed for better DNC training and inference efficiency. The proposed method consists of two phases: i) a flip DNC (F-DNC) is exploited to rank most likely flip positions for multi-bit flipping; ii) if decoding still fails, a flip-validate DNC (FV-DNC) is used to re-select error bit positions for successive flip decoding trials. Supervised training methods are designed accordingly for the two DNCs. Simulation results show that proposed DNC-aided SCL-Flip (DNC-SCLF) decoding demonstrates up to 0.34dB coding gain improvement or 54.2 reduction in average number of decoding attempts compared to prior works.
Continual Learning of Visual Concepts for Robots through Limited Supervision
Venue: arXiv cs.RO 2021
Year: 2021

For many real-world robotics applications, robots need to continually adapt and learn new concepts. Further, robots need to learn through limited data because of scarcity of labeled data in the real-world environments. To this end, my research focuses on developing robots that continually learn in dynamic unseen environments/scenarios, learn from limited human supervision, remember previously learned knowledge and use that knowledge to learn new concepts. I develop machine learning models that not only produce State-of-the-results on benchmark datasets but also allow robots to learn new objects and scenes in unconstrained environments which lead to a variety of novel robotics applications.
How Good is a Video Summary? A New Benchmarking Dataset and Evaluation Framework Towards Realistic Video Summarization
Venue: arXiv cs.CV 2021
Year: 2021

Automatic video summarization is still an unsolved problem due to several challenges. The currently available datasets either have very short videos or have few long videos of only a particular type. We introduce a new benchmarking video dataset called VISIOCITY (VIdeo SummarIzatiOn based on Continuity, Intent and DiversiTY) which comprises of longer videos across six different categories with dense concept annotations capable of supporting different flavors of video summarization and other vision problems. For long videos, human reference summaries necessary for supervised video summarization techniques are difficult to obtain. We explore strategies to automatically generate multiple reference summaries from indirect ground truth present in VISIOCITY. We show that these summaries are at par with human summaries. We also present a study of different desired characteristics of a good summary and demonstrate how it is normal to have two good summaries with different characteristics. Thus we argue that evaluating a summary against one or more human summaries and using a single measure has its shortcomings. We propose an evaluation framework for better quantitative assessment of summary quality which is closer to human judgment. Lastly, we present insights into how a model can be enhanced to yield better summaries. Sepcifically, when multiple diverse ground truth summaries can exist, learning from them individually and using a combination of loss functions measuring different characteristics is better than learning from a single combined (oracle) ground truth summary using a single loss function. We demonstrate the effectiveness of doing so as compared to some of the representative state of the art techniques tested on VISIOCITY. We release VISIOCITY as a benchmarking dataset and invite researchers to test the effectiveness of their video summarization algorithms on VISIOCITY.
Study of Graph Theory, Distributed Average Consensus Algorithm and Centralized Algorithm
Venue: arXiv cs.DM 2021
Year: 2021

In this paper, we hope to bring closer graph theory and consensus algorithms. Firstly, we give a brief introduction to graph theory by listing a concise definition. Then we analyze and visualize some commonly used graphs. Secondly, we record an overview of the consensus algorithm, including its structure and significance. We then visualize and explore how the distributed average scheme will reach consensus node values for different probability distributions. Thirdly, we introduce a centralized algorithm about its organizations and functions. Using nodes with other distributions, we analyze how a centralized algorithm will use those nodes to affect the initialize ones. Fourthly, we bring several root-finding algorithms as a supplement to this paper. Finally, we derive conclusions and list substantial mathematical deviations and algorithms.
El Volumen Louder Por Favor: Code-switching in Task-oriented Semantic Parsing
Venue: arXiv cs.CL 2021
Year: 2021

Being able to parse code-switched (CS) utterances, such as Spanish+English or Hindi+English, is essential to democratize task-oriented semantic parsing systems for certain locales. In this work, we focus on Spanglish (Spanish+English) and release a dataset, CSTOP, containing 5800 CS utterances alongside their semantic parses. We examine the CS generalizability of various Cross-lingual (XL) models and exhibit the advantage of pre-trained XL language models when data for only one language is present. As such, we focus on improving the pre-trained models for the case when only English corpus alongside either zero or a few CS training instances are available. We propose two data augmentation methods for the zero-shot and the few-shot settings: fine-tune using translate-and-align and augment using a generation model followed by match-and-filter. Combining the few-shot setting with the above improvements decreases the initial 30-point accuracy gap between the zero-shot and the full-data settings by two thirds.
Modern Machine and Deep Learning Systems as a way to achieve Man-Computer Symbiosis
Venue: arXiv cs.LG 2021
Year: 2021

Man-Computer Symbiosis (MCS) was originally envisioned by the famous computer pioneer J.C.R. Licklider in 1960, as a logical evolution of the then inchoate relationship between computer and humans. In his paper, Licklider provided a set of criteria by which to judge if a Man-Computer System is a symbiotic one, and also provided some predictions about such systems in the near and far future. Since then, innovations in computer networks and the invention of the Internet were major developments towards that end. However, with most systems based on conventional logical algorithms, many aspects of Licklider's MCS remained unfulfilled. This paper explores the extent to which modern machine learning systems in general, and deep learning ones in particular best exemplify MCS systems, and why they are the prime contenders to achieve a true Man-Computer Symbiosis as described by Licklider in his original paper in the future. The case for deep learning is built by illustrating each point of the original criteria as well as the criteria laid by subsequent research into MCS systems, with specific examples and applications provided to strengthen the arguments. The efficacy of deep neural networks in achieving Artificial General Intelligence, which would be the perfect version of an MCS system is also explored.
Differential Privacy for Industrial Internet of Things: Opportunities, Applications and Challenges
Venue: arXiv cs.CR 2021
Year: 2021

The development of Internet of Things (IoT) brings new changes to various fields. Particularly, industrial Internet of Things (IIoT) is promoting a new round of industrial revolution. With more applications of IIoT, privacy protection issues are emerging. Specially, some common algorithms in IIoT technology such as deep models strongly rely on data collection, which leads to the risk of privacy disclosure. Recently, differential privacy has been used to protect user-terminal privacy in IIoT, so it is necessary to make in-depth research on this topic. In this paper, we conduct a comprehensive survey on the opportunities, applications and challenges of differential privacy in IIoT. We firstly review related papers on IIoT and privacy protection, respectively. Then we focus on the metrics of industrial data privacy, and analyze the contradiction between data utilization for deep models and individual privacy protection. Several valuable problems are summarized and new research ideas are put forward. In conclusion, this survey is dedicated to complete comprehensive summary and lay foundation for the follow-up researches on industrial differential privacy.
Malware Detection Using Frequency Domain-Based Image Visualization and Deep Learning
Venue: arXiv cs.CR 2021
Year: 2021

We propose a novel method to detect and visualize malware through image classification. The executable binaries are represented as grayscale images obtained from the count of N-grams (N=2) of bytes in the Discrete Cosine Transform (DCT) domain and a neural network is trained for malware detection. A shallow neural network is trained for classification, and its accuracy is compared with deep-network architectures such as ResNet that are trained using transfer learning. Neither dis-assembly nor behavioral analysis of malware is required for these methods. Motivated by the visual similarity of these images for different malware families, we compare our deep neural network models with standard image features like GIST descriptors to evaluate the performance. A joint feature measure is proposed to combine different features using error analysis to get an accurate ensemble model for improved classification performance. A new dataset called MaleX which contains around 1 million malware and benign Windows executable samples is created for large-scale malware detection and classification experiments. Experimental results are quite promising with 96% binary classification accuracy on MaleX. The proposed model is also able to generalize well on larger unseen malware samples and the results compare favorably with state-of-the-art static analysis-based malware detection algorithms.
xLumi: Payment Channel Protocol and Off-chain Payment in Blockchain Contract Systems
Venue: arXiv cs.CR 2021
Year: 2021

In this paper, we introduce Super Luminal ("xLumi"), a new payment channel protocol for blockchain systems. xLumi is a simple unidirectional payment channel that can be extended to a bidirectional payment channel or a complete network. xLumi guarantees the security of the payment channel's funds by using a simple set of mathematical rules that can be easily implemented on any blockchain with the necessary infrastructure. We also give the detailed implementation methods of this idea using V Systems contract systems in this paper.
A Survey on Data Plane Programming with P4: Fundamentals, Advances, and Applied Research
Venue: arXiv cs.NI 2021
Year: 2021

Programmable data planes allow users to define their own data plane algorithms for network devices including appropriate data plane application programming interfaces (APIs) which may be leveraged by user-defined software-defined networking (SDN) control. This offers great flexibility for network customization, be it for specialized, commercial appliances, e.g., in 5G or data center networks, or for rapid prototyping in industrial and academic research. Programming protocol-independent packet processors (P4) has emerged as the currently most widespread abstraction, programming language, and concept for data plane programming. It is developed and standardized by an open community, and it is supported by various software and hardware platforms. In the first part of this paper we give a tutorial of data plane programming models, the P4 programming language, architectures, compilers, targets, and data plane APIs. We also consider research efforts to advance P4 technology. In the second part, we categorize a large body of literature of P4-based applied research into different research domains, summarize the contributions of these papers, and extract prototypes, target platforms, and source code availability. For each research domain, we analyze how the reviewed works benefit from P4's core features. Finally, we discuss potential next steps based on our findings.
Measuring Decentralization in Bitcoin and Ethereum using Multiple Metrics and Granularities
Venue: arXiv cs.CR 2021
Year: 2021

Decentralization has been widely acknowledged as a core virtue of blockchains. However, in the past, there have been few measurement studies on measuring and comparing the actual level of decentralization between existing blockchains using multiple metrics and granularities. This paper presents a new comparison study of the degree of decentralization in Bitcoin and Ethereum, the two most prominent blockchains, with various decentralization metrics and different granularities within the time dimension. Specifically, we measure the degree of decentralization in the two blockchains during 2019 by computing the distribution of mining power with three metrics (Gini coefficient, Shannon entropy, and Nakamoto coefficient) as well as three granularities (days, weeks, and months). Our measurement results with different metrics and granularities reveal the same trend that, compared with each other, the degree of decentralization in Bitcoin is higher, while the degree of decentralization in Ethereum is more stable. To obtain the cross-interval information missed in the fixed window based measurements, we propose the sliding window based measurement approach. The corresponding results demonstrate that the use of sliding windows could reveal additional cross-interval information overlooked by the fixed window based measurements, thus enhancing the effectiveness of measuring decentralization in terms of continuous trends and abnormal situations. We believe that the methodologies and findings in this paper can facilitate future studies of decentralization in blockchains.
The Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games
Venue: arXiv cs.HC 2021
Year: 2021

What if emotion could be captured in a general and subject-agnostic fashion? Is it possible, for instance, to design general-purpose representations that detect affect solely from the pixels and audio of a human-computer interaction video? In this paper we address the above questions by evaluating the capacity of deep learned representations to predict affect by relying only on audiovisual information of videos. We assume that the pixels and audio of an interactive session embed the necessary information required to detect affect. We test our hypothesis in the domain of digital games and evaluate the degree to which deep classifiers and deep preference learning algorithms can learn to predict the arousal of players based only on the video footage of their gameplay. Our results from four dissimilar games suggest that general-purpose representations can be built across games as the arousal models obtain average accuracies as high as 85% using the challenging leave-one-video-out cross-validation scheme. The dissimilar audiovisual characteristics of the tested games showcase the strengths and limitations of the proposed method.
Enhancing Application Performance by Memory Partitioning in Android Platforms
Venue: arXiv cs.OS 2021
Year: 2021

This paper suggests a new memory partitioning scheme that can enhance process lifecycle, while avoiding Low Memory Killer and Out-of-Memory Killer operations on mobile devices. Our proposed scheme offers the complete concept of virtual memory nodes in operating systems of Android devices.
A program logic for fresh name generation
Venue: arXiv cs.LO 2021
Year: 2021

We present a program logic for Pitts and Stark's ν-calculus, an extension of the call-by-value simply-typed λ-calculus with a mechanism for the generation of fresh names. Names can be compared for (in)-equality, producing programs with subtle observable properties. Hidden names produced by interactions between generation and abstraction are captured logically with a second-order quantifier over type contexts. We illustrate usage of the logic through reasoning about well-known difficult cases from the literature.
Developing emotion recognition for video conference software to support people with autism
Venue: arXiv cs.CV 2021
Year: 2021

We develop an emotion recognition software for the use with a video conference software for autistic individuals which are unable to recognize emotions properly. It can get an image out of the video stream, detect the emotion in it with the help of a neural network and display the prediction to the user. The network is trained on facial landmark features. The software is fully modular to support adaption to different video conference software, programming languages and implementations.
Adversarial Vulnerability of Active Transfer Learning
Venue: arXiv cs.LG 2021
Year: 2021

Two widely used techniques for training supervised machine learning models on small datasets are Active Learning and Transfer Learning. The former helps to optimally use a limited budget to label new data. The latter uses large pre-trained models as feature extractors and enables the design of complex, non-linear models even on tiny datasets. Combining these two approaches is an effective, state-of-the-art method when dealing with small datasets. In this paper, we share an intriguing observation: Namely, that the combination of these techniques is particularly susceptible to a new kind of data poisoning attack: By adding small adversarial noise on the input, it is possible to create a collision in the output space of the transfer learner. As a result, Active Learning algorithms no longer select the optimal instances, but almost exclusively the ones injected by the attacker. This allows an attacker to manipulate the active learner to select and include arbitrary images into the data set, even against an overwhelming majority of unpoisoned samples. We show that a model trained on such a poisoned dataset has a significantly deteriorated performance, dropping from 86\% to 34\% test accuracy. We evaluate this attack on both audio and image datasets and support our findings empirically. To the best of our knowledge, this weakness has not been described before in literature.
Exploring Transfer Learning on Face Recognition of Dark Skinned, Low Quality and Low Resource Face Data
Venue: arXiv cs.CV 2021
Year: 2021

There is a big difference in the tone of color of skin between dark and light skinned people. Despite this fact, most face recognition tasks almost all classical state-of-the-art models are trained on datasets containing an overwhelming majority of light skinned face images. It is tedious to collect a huge amount of data for dark skinned faces and train a model from scratch. In this paper, we apply transfer learning on VGGFace to check how it works on recognising dark skinned mainly Ethiopian faces. The dataset is of low quality and low resource. Our experimental results show above 95\% accuracy which indicates that transfer learning in such settings works.