OpenRaiser/Intern-Atlas huggingface dataset:
Partial Information Decomposition of Boolean Functions: a Fourier Analysis perspective
Venue: arXiv cs.IT 2020
Year: 2020

Partial information decomposition (PID) partitions the information that a set of sources has about a target variable into synergistic, unique, and redundant contributions. This information-theoretic tool has recently attracted attention due to its potential to characterize the information processing in multivariate systems. However, the PID framework still lacks a solid and intuitive interpretation of its information components. In the aim to improve the understanding of PID components, we focus here on Boolean gates, a much-studied type of source-target mechanisms. Boolean gates have been extensively characterised via Fourier analysis which coefficients have been related to interesting properties of the functions defining the gates. In this paper, we establish for Boolean gates mechanisms a relation between their PID components and Fourier coefficients.
A Case for a Currencyless Economy Based on Bartering with Smart Contracts
Venue: arXiv cs.CY 2020
Year: 2020

We suggest the re-introduction of bartering to create a cryptocurrencyless, currencyless, and moneyless economy segment. We contend that a barter economy would benefit enterprises, individuals, governments and societies. For instance, the availability of an online peer-to-peer barter marketplace would convert ordinary individuals into potential traders of both tangible and digital items and services. For example, they will be able to barter files and data that they collect. Equally motivating, they will be able to barter and re-introduce to the economy items that they no longer need such as, books, garden tools, and bikes which are normally kept and wasted in garages and sheds. We argue that most of the pieces of technology needed for building a barter system are now available, including blockchains, smart contracts, cryptography, secure multiparty computations and fair exchange protocols. However, additional research is needed to refine and integrate the pieces together. We discuss potential research directions.
Computational Skills by Stealth in Secondary School Data Science
Venue: arXiv cs.CY 2020
Year: 2020

The unprecedented growth in the availability of data of all types and qualities and the emergence of the field of data science has provided an impetus to finally realizing the implementation of the full breadth of the Nolan and Temple Lang proposed integration of computing concepts into statistics curricula at all levels in statistics and new data science programs and courses. Moreover, data science, implemented carefully, opens accessible pathways to stem for students for whom neither mathematics nor computer science are natural affinities, and who would traditionally be excluded. We discuss a proposal for the stealth development of computational skills in students' first exposure to data science through careful, scaffolded exposure to computation and its power. The intent of this approach is to support students, regardless of interest and self-efficacy in coding, in becoming data-driven learners, who are capable of asking complex questions about the world around them, and then answering those questions through the use of data-driven inquiry. This discussion is presented in the context of the International Data Science in Schools Project which recently published computer science and statistics consensus curriculum frameworks for a two-year secondary school data science program, designed to make data science accessible to all.
A Tendermint Light Client
Venue: arXiv cs.DC 2020
Year: 2020

In Tendermint blockchains, the proof-of-stake mechanism and the underlying consensus algorithm entail a dynamic fault model that implies that the active validators (nodes that sign blocks) may change over time, and a quorum of these validators is assumed to be correct only for a limited period of time (called trusting period). The changes of the validator set are under control of the blockchain application, and are committed in every block. In order to check what is the state of the blockchain application at some height h, one needs to know the validator set at that height so that one can verify the corresponding digital signatures and hashes. A naive way of determining the validator set for height h requires one to: (i) download all blocks before h, (ii) verify blocks by checking digital signatures and hashes and (iii) execute the corresponding transactions so the changes in the validator sets are reproduced. This can potentially be very slow and computationally and data intensive. In this paper we formalize the dynamic fault model imposed by Tendermint, and describe a light client protocol that allows to check the state of the blockchain application that, in realistic settings, reduces significantly the amount of data needed to be downloaded, and the number of required computationally expensive signature verification operations. In addition to mathematical proofs, we have formalized the light client protocol in TLA+, and checked safety and liveness with the APALACHE model checker.
Modurec: Recommender Systems with Feature and Time Modulation
Venue: arXiv cs.IR 2020
Year: 2020

Current state of the art algorithms for recommender systems are mainly based on collaborative filtering, which exploits user ratings to discover latent factors in the data. These algorithms unfortunately do not make effective use of other features, which can help solve two well identified problems of collaborative filtering: cold start (not enough data is available for new users or products) and concept shift (the distribution of ratings changes over time). To address these problems, we propose Modurec: an autoencoder-based method that combines all available information using the feature-wise modulation mechanism, which has demonstrated its effectiveness in several fields. While time information helps mitigate the effects of concept shift, the combination of user and item features improve prediction performance when little data is available. We show on Movielens datasets that these modifications produce state-of-the-art results in most evaluated settings compared with standard autoencoder-based methods and other collaborative filtering approaches.
Differentiable Implicit Layers
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we introduce an efficient backpropagation scheme for non-constrained implicit functions. These functions are parametrized by a set of learnable weights and may optionally depend on some input; making them perfectly suitable as a learnable layer in a neural network. We demonstrate our scheme on different applications: (i) neural ODEs with the implicit Euler method, and (ii) system identification in model predictive control.
An Investigation on Different Underlying Quantization Schemes for Pre-trained Language Models
Venue: arXiv cs.CL 2020
Year: 2020

Recently, pre-trained language models like BERT have shown promising performance on multiple natural language processing tasks. However, the application of these models has been limited due to their huge size. To reduce its size, a popular and efficient way is quantization. Nevertheless, most of the works focusing on BERT quantization adapted primary linear clustering as the quantization scheme, and few works try to upgrade it. That limits the performance of quantization significantly. In this paper, we implement k-means quantization and compare its performance on the fix-precision quantization of BERT with linear quantization. Through the comparison, we verify that the effect of the underlying quantization scheme upgrading is underestimated and there is a huge development potential of k-means quantization. Besides, we also compare the two quantization schemes on ALBERT models to explore the robustness differences between different pre-trained models.
On the performance of a highly-scalable Computational Fluid Dynamics code on AMD, ARM and Intel processors
Venue: arXiv cs.DC 2020
Year: 2020

No area of computing is hungrier for performance than High Performance Computing (HPC), the demands of which continue to be a major driver for processor performance and adoption of accelerators, and also advances in memory, storage, and networking technologies. A key feature of the Intel processor domination of the past decade has been the extensive adoption of GPUs as coprocessors, whilst more recent developments have seen the increased availability of a number of CPU processors, including the novel ARM-based chips. This paper analyses the performance and scalability of a state-of-the-art Computational Fluid Dynamics (CFD) code on three HPC cluster systems equipped with AMD EPYC-Rome (EPYC, 4096 cores), ARM-based Marvell ThunderX2 (TX2, 8192 cores) and Intel Skylake (SKL, 8000 cores) processors. Three benchmark cases are designed with increasing computation-to-communication ratio and numerical complexity, namely lid-driven cavity flow, Taylor-Green vortex and a travelling solitary wave using the level-set method, adopted with $4^{th}$-order central-differences or a $5^{th}$-order WENO scheme. Our results show that the EPYC cluster delivers the best code performance for all the setups under consideration. In the first two benchmarks, the SKL cluster demonstrates faster computing times than the TX2 system, whilst in the solitary wave simulations, the TX2 cluster achieves good scalability and similar performance to the EPYC system, both improving on that obtained with the SKL cluster. These results suggest that while the Intel SKL cores deliver the best strong scalability, the associated cluster performance is lower compared to the EPYC system. The TX2 cluster performance is promising considering its recent addition to the HPC portfolio.
The EOS Decision and Length Extrapolation
Venue: arXiv cs.CL 2020
Year: 2020

Extrapolation to unseen sequence lengths is a challenge for neural generative models of language. In this work, we characterize the effect on length extrapolation of a modeling decision often overlooked: predicting the end of the generative process through the use of a special end-of-sequence (EOS) vocabulary item. We study an oracle setting - forcing models to generate to the correct sequence length at test time - to compare the length-extrapolative behavior of networks trained to predict EOS (+EOS) with networks not trained to (-EOS). We find that -EOS substantially outperforms +EOS, for example extrapolating well to lengths 10 times longer than those seen at training time in a bracket closing task, as well as achieving a 40% improvement over +EOS in the difficult SCAN dataset length generalization task. By comparing the hidden states and dynamics of -EOS and +EOS models, we observe that +EOS models fail to generalize because they (1) unnecessarily stratify their hidden states by their linear position is a sequence (structures we call length manifolds) or (2) get stuck in clusters (which we refer to as length attractors) once the EOS token is the highest-probability prediction.
Towards Increasing Trust In Expert Evidence Derived From Malware Forensic Tools
Venue: arXiv cs.CY 2020
Year: 2020

Following a series of high profile miscarriages of justice in the UK linked to questionable expert evidence, the post of the Forensic Science Regulator was created in 2008. The main objective of this role is to improve the standard of practitioner competences and forensic procedures. One of the key strategies deployed to achieve this is the push to incorporate a greater level of scientific conduct in the various fields of forensic practice. Currently there is no statutory requirement for practitioners to become accredited to continue working with the Criminal Justice System of England and Wales. However, the Forensic Science Regulator is lobbying the UK Government to make this mandatory. This paper focuses upon the challenge of incorporating a scientific methodology to digital forensic investigations where malicious software ('malware') has been identified. One aspect of such a methodology is the approach followed to both select and evaluate the tools used to perform dynamic malware analysis during an investigation. Based on the literature, legal, regulatory and practical needs we derive a set of requirements to address this challenge. We present a framework, called the 'Malware Analysis Tool Evaluation Framework' (MATEF), to address this lack of methodology to evaluate software tools used to perform dynamic malware analysis during investigations involving malware and discuss how it meets the derived requirements.
Learning Improvised Chatbots from Adversarial Modifications of Natural Language Feedback
Venue: arXiv cs.CL 2020
Year: 2020

The ubiquitous nature of chatbots and their interaction with users generate an enormous amount of data. Can we improve chatbots using this data? A self-feeding chatbot improves itself by asking natural language feedback when a user is dissatisfied with its response and uses this feedback as an additional training sample. However, user feedback in most cases contains extraneous sequences hindering their usefulness as a training sample. In this work, we propose a generative adversarial model that converts noisy feedback into a plausible natural response in a conversation. The generator's goal is to convert the feedback into a response that answers the user's previous utterance and to fool the discriminator which distinguishes feedback from natural responses. We show that augmenting original training data with these modified feedback responses improves the original chatbot performance from 69.94% to 75.96% in ranking correct responses on the Personachat dataset, a large improvement given that the original model is already trained on 131k samples.
Wukong: A Scalable and Locality-Enhanced Framework for Serverless Parallel Computing
Venue: arXiv cs.DC 2020
Year: 2020

Serverless computing is increasingly being used for parallel computing, which have traditionally been implemented as stateful applications. Executing complex, burst-parallel, directed acyclic graph (DAG) jobs poses a major challenge for serverless execution frameworks, which will need to rapidly scale and schedule tasks at high throughput, while minimizing data movement across tasks. We demonstrate that, for serverless parallel computations, decentralized scheduling enables scheduling to be distributed across Lambda executors that can schedule tasks in parallel, and brings multiple benefits, including enhanced data locality, reduced network I/Os, automatic resource elasticity, and improved cost effectiveness. We describe the implementation and deployment of our new serverless parallel framework, called Wukong, on AWS Lambda. We show that Wukong achieves near-ideal scalability, executes parallel computation jobs up to 68.17x faster, reduces network I/O by multiple orders of magnitude, and achieves 92.96% tenant-side cost savings compared to numpywren.
Overview of C-ITS Deployment Projects in Europe and USA
Venue: arXiv cs.CY 2020
Year: 2020

Cooperative Intelligent Transportation Systems (C-ITS) are technologies that enable vehicles to communicate with each other and with the road infrastructure. These innovative technologies enable road users and traffic managers to share useful information, assisting the coordination of their actions. During the last years various initiatives providing policy rules for C-ITS deployment and a large number of projects demonstrating C-ITS implementation have taken place in Europe and USA. However, the identification of the status of C-ITS deployment remains ambiguous at binational level. The purpose of this paper is to provide an overview of the European and US milestones, that have been reached so far in the field of C-ITS, by identifying and reporting the policy framework, as well as the projects concerning C-ITS deployment in Europe and USA.
Online Learning with Vector Costs and Bandits with Knapsacks
Venue: arXiv cs.LG 2020
Year: 2020

We introduce online learning with vector costs (\OLVCp) where in each time step $t \in \{1,\ldots, T\}$, we need to play an action $i \in \{1,\ldots,n\}$ that incurs an unknown vector cost in $[0,1]^{d}$. The goal of the online algorithm is to minimize the $\ell_p$ norm of the sum of its cost vectors. This captures the classical online learning setting for $d=1$, and is interesting for general $d$ because of applications like online scheduling where we want to balance the load between different machines (dimensions). We study \OLVCp in both stochastic and adversarial arrival settings, and give a general procedure to reduce the problem from $d$ dimensions to a single dimension. This allows us to use classical online learning algorithms in both full and bandit feedback models to obtain (near) optimal results. In particular, we obtain a single algorithm (up to the choice of learning rate) that gives sublinear regret for stochastic arrivals and a tight $O(\min\{p, \log d\})$ competitive ratio for adversarial arrivals. The \OLVCp problem also occurs as a natural subproblem when trying to solve the popular Bandits with Knapsacks (\BwK) problem. This connection allows us to use our \OLVCp techniques to obtain (near) optimal results for \BwK in both stochastic and adversarial settings. In particular, we obtain a tight $O(\log d \cdot \log T)$ competitive ratio algorithm for adversarial \BwK, which improves over the $O(d \cdot \log T)$ competitive ratio algorithm of Immorlica et al. [FOCS'19].
Photovoltaic module segmentation and thermal analysis tool from thermal images
Venue: arXiv cs.CV 2020
Year: 2020

The growing interest in the use of clean energy has led to the construction of increasingly large photovoltaic systems. Consequently, monitoring the proper functioning of these systems has become a highly relevant issue.In this paper, automatic detection, and analysis of photovoltaic modules are proposed. To perform the analysis, a module identification step, based on a digital image processing algorithm, is first carried out. This algorithm consists of image enhancement (contrast enhancement, noise reduction, etc.), followed by segmentation of the photovoltaic module. Subsequently, a statistical analysis based on the temperature values of the segmented module is performed.Besides, a graphical user interface has been designed as a potential tool that provides relevant information of the photovoltaic modules.
Complexity Measures on the Symmetric Group and Beyond
Venue: arXiv cs.CC 2020
Year: 2020

We extend the definitions of complexity measures of functions to domains such as the symmetric group. The complexity measures we consider include degree, approximate degree, decision tree complexity, sensitivity, block sensitivity, and a few others. We show that these complexity measures are polynomially related for the symmetric group and for many other domains. To show that all measures but sensitivity are polynomially related, we generalize classical arguments of Nisan and others. To add sensitivity to the mix, we reduce to Huang's sensitivity theorem using "pseudo-characters", which witness the degree of a function. Using similar ideas, we extend the characterization of Boolean degree 1 functions on the symmetric group due to Ellis, Friedgut and Pilpel to the perfect matching scheme. As another application of our ideas, we simplify the characterization of maximum-size $t$-intersecting families in the symmetric group and the perfect matching scheme.
AI-based BMI Inference from Facial Images: An Application to Weight Monitoring
Venue: arXiv cs.CV 2020
Year: 2020

Self-diagnostic image-based methods for healthy weight monitoring is gaining increased interest following the alarming trend of obesity. Only a handful of academic studies exist that investigate AI-based methods for Body Mass Index (BMI) inference from facial images as a solution to healthy weight monitoring and management. To promote further research and development in this area, we evaluate and compare the performance of five different deep-learning based Convolutional Neural Network (CNN) architectures i.e., VGG19, ResNet50, DenseNet, MobileNet, and lightCNN for BMI inference from facial images. Experimental results on the three publicly available BMI annotated facial image datasets assembled from social media, namely, VisualBMI, VIP-Attributes, and Bollywood datasets, suggest the efficacy of the deep learning methods in BMI inference from face images with minimum Mean Absolute Error (MAE) of $1.04$ obtained using ResNet50.
Development and Validation of a Data Fusion Algorithm with Low-Cost Inertial Measurement Units to Analyze Shoulder Movements in Manual Workers
Venue: arXiv cs.RO 2020
Year: 2020

Work-related upper extremity musculoskeletal disorders (WRUED) are a major problem in modern societies as they affect the quality of life of workers and lead to absenteeism and productivity loss. According to studies performed in North America and Western Europe, their prevalence has increased in the last few decades. This challenge calls for improvements in prevention methods. One avenue is through the development of wearable sensor systems to analyze worker's movements and provide feedback to workers and/or clinicians. Such systems could decrease the physical work demands and ultimately prevent musculoskeletal disorders. This paper presents the development and validation of a data fusion algorithm for inertial measurement units to analyze worker's arm elevation. The algorithm was implemented on two commercial sensor systems (Actigraph GT9X and LSM9DS1) and results were compared with the data fusion results from a validated commercial sensor (XSens MVN system). Cross-correlation analyses [r], root-mean-square error (RMSE) and average absolute error of estimate were used to establish the construct validity of the algorithm. Five subjects each performed ten different arm elevation tasks. The results show that the algorithm is valid to evaluate shoulder movements with high correlations between the results of the two different sensors and the commercial sensor (0.900-0.998) and relatively low RMSE value for the ten tasks (1.66-11.24°). The proposed data fusion algorithm could thus be used to estimate arm elevation.
Causal Inference in the Presence of Interference in Sponsored Search Advertising
Venue: arXiv cs.AI 2020
Year: 2020

In classical causal inference, inferring cause-effect relations from data relies on the assumption that units are independent and identically distributed. This assumption is violated in settings where units are related through a network of dependencies. An example of such a setting is ad placement in sponsored search advertising, where the clickability of a particular ad is potentially influenced by where it is placed and where other ads are placed on the search result page. In such scenarios, confounding arises due to not only the individual ad-level covariates but also the placements and covariates of other ads in the system. In this paper, we leverage the language of causal inference in the presence of interference to model interactions among the ads. Quantification of such interactions allows us to better understand the click behavior of users, which in turn impacts the revenue of the host search engine and enhances user satisfaction. We illustrate the utility of our formalization through experiments carried out on the ad placement system of the Bing search engine.