OpenRaiser/Intern-Atlas huggingface dataset:
Proper Learning, Helly Number, and an Optimal SVM Bound
Venue: arXiv cs.LG 2020
Year: 2020

The classical PAC sample complexity bounds are stated for any Empirical Risk Minimizer (ERM) and contain an extra logarithmic factor $\log(1/ε)$ which is known to be necessary for ERM in general. It has been recently shown by Hanneke (2016) that the optimal sample complexity of PAC learning for any VC class C is achieved by a particular improper learning algorithm, which outputs a specific majority-vote of hypotheses in C. This leaves the question of when this bound can be achieved by proper learning algorithms, which are restricted to always output a hypothesis from C. In this paper we aim to characterize the classes for which the optimal sample complexity can be achieved by a proper learning algorithm. We identify that these classes can be characterized by the dual Helly number, which is a combinatorial parameter that arises in discrete geometry and abstract convexity. In particular, under general conditions on C, we show that the dual Helly number is bounded if and only if there is a proper learner that obtains the optimal joint dependence on $ε$ and $δ$. As further implications of our techniques we resolve a long-standing open problem posed by Vapnik and Chervonenkis (1974) on the performance of the Support Vector Machine by proving that the sample complexity of SVM in the realizable case is $Θ((n/ε)+(1/ε)\log(1/δ))$, where $n$ is the dimension. This gives the first optimal PAC bound for Halfspaces achieved by a proper learning algorithm, and moreover is computationally efficient.
Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model
Venue: arXiv cs.CL 2020
Year: 2020

Studies on grammatical error correction (GEC) have reported the effectiveness of pretraining a Seq2Seq model with a large amount of pseudodata. However, this approach requires time-consuming pretraining for GEC because of the size of the pseudodata. In this study, we explore the utility of bidirectional and auto-regressive transformers (BART) as a generic pretrained encoder-decoder model for GEC. With the use of this generic pretrained model for GEC, the time-consuming pretraining can be eliminated. We find that monolingual and multilingual BART models achieve high performance in GEC, with one of the results being comparable to the current strong results in English GEC. Our implementations are publicly available at GitHub (https://github.com/Katsumata420/generic-pretrained-GEC).
The never ending war in the stack and the reincarnation of ROP attacks
Venue: arXiv cs.CR 2020
Year: 2020

Return Oriented Programming (ROP) is a technique by which an attacker can induce arbitrary behavior inside a vulnerable program without injecting a malicious code. The continues failure of the currently deployed defenses against ROP has made it again one of the most powerful memory corruption attacks. ROP is also considered as one of the most flexible attacks, its level of flexibility, unlike other code reuse attacks, can reach the Turing completeness. Several efforts have been undertaken to study this threat and to propose better defense mechanisms (mitigation or prevention), yet the majority of them are not deeply reviewed nor officially implemented.Furthermore, similar studies show that the techniques proposed to prevent ROP-based exploits usually yield a high false-negative rate and a higher false-positive rate, not to mention the overhead that they introduce into the protected program. The first part of this research work aims at providing an in-depth analysis of the currently available anti-ROP solutions (deployed and proposed), focusing on inspecting their defense logic and summarizing their weaknesses and problems. The second part of this work aims at introducing our proposed Indicators Of Compromise (IOCs) that could be used to improve the detection rate of ROP attacks. The three suggested indicators could detect these attacks at run-time by checking the presence of some artifacts during the execution of the targeted program.
AutoSUM: Automating Feature Extraction and Multi-user Preference Simulation for Entity Summarization
Venue: arXiv cs.IR 2020
Year: 2020

Withthegrowthofknowledgegraphs, entity descriptions are becoming extremely lengthy. Entity summarization task, aiming to generate diverse, comprehensive, and representative summaries for entities, has received increasing interest recently. In most previous methods, features are usually extracted by the handcrafted templates. Then the feature selection and multi-user preference simulation take place, depending too much on human expertise. In this paper, a novel integration method called AutoSUM is proposed for automatic feature extraction and multi-user preference simulation to overcome the drawbacks of previous methods. There are two modules in AutoSUM: extractor and simulator. The extractor module operates automatic feature extraction based on a BiLSTM with a combined input representation including word embeddings and graph embeddings. Meanwhile, the simulator module automates multi-user preference simulation based on a well-designed two-phase attention mechanism (i.e., entity-phase attention and user-phase attention). Experimental results demonstrate that AutoSUM produces state-of-the-art performance on two widely used datasets (i.e., DBpedia and LinkedMDB) in both F-measure and MAP.
Dynamic Cache Management In Content Delivery Networks
Venue: arXiv cs.NI 2020
Year: 2020

The importance of content delivery networks (CDN) continues to rise with the exponential increase in the generation and consumption of electronic media. In order to ensure a high quality of experience, CDNs often deploy cache servers that are capable of storing some of the popular files close to the user. Such edge caching solutions not only increase the content availability, but also result in higher download rates and lower latency at the user. We consider the problem of content placement from an optimization perspective. Different from the classical eviction-based algorithms, the present work formulates the content placement problem from an optimization perspective and puts forth an online algorithm for the same. In contrast to the existing optimization-based solutions, the proposed algorithm is incremental and incurs very low computation cost, while yielding storage allocations that are provably near-optimal. The proposed algorithm can handle time varying content popularity, thereby obviating the need for periodically estimating demand distribution. Using synthetic and real IPTV data, we show that the proposed policies outperform all the state of art caching techniques in terms of various metrics.
Multi-Margin based Decorrelation Learning for Heterogeneous Face Recognition
Venue: arXiv cs.CV 2020
Year: 2020

Heterogeneous face recognition (HFR) refers to matching face images acquired from different domains with wide applications in security scenarios. This paper presents a deep neural network approach namely Multi-Margin based Decorrelation Learning (MMDL) to extract decorrelation representations in a hyperspherical space for cross-domain face images. The proposed framework can be divided into two components: heterogeneous representation network and decorrelation representation learning. First, we employ a large scale of accessible visual face images to train heterogeneous representation network. The decorrelation layer projects the output of the first component into decorrelation latent subspace and obtains decorrelation representation. In addition, we design a multi-margin loss (MML), which consists of quadruplet margin loss (QML) and heterogeneous angular margin loss (HAML), to constrain the proposed framework. Experimental results on two challenging heterogeneous face databases show that our approach achieves superior performance on both verification and recognition tasks, comparing with state-of-the-art methods.
On Irrelevance of Attributes in Flexible Prediction
Venue: arXiv cs.AI 2020
Year: 2020

This paper analyses properties of conceptual hierarchy obtained via incremental concept formation method called "flexible prediction" in order to determine what kind of "relevance" of participating attributes may be requested for meaningful conceptual hierarchy. The impact of selection of simple and combined attributes, of scaling and of distribution of individual attributes and of correlation strengths among them is investigated. Paradoxically, both: attributes weakly and strongly related with other attributes have deteriorating impact onto the overall classification. Proper construction of derived attributes as well as selection of scaling of individual attributes strongly influences the obtained concept hierarchy. Attribute density of distribution seems to influence the classification weakly It seems also, that concept hierarchies (taxonomies) reflect a compromise between the data and our interests in some objective truth about the data. To obtain classifications more suitable for one's purposes, breaking the symmetry among attributes (by dividing them into dependent and independent and applying differing evaluation formulas for their contribution) is suggested. Both continuous and discrete variables are considered. Some methodologies for the former are considered.
Eye Gaze Controlled Robotic Arm for Persons with SSMI
Venue: arXiv cs.HC 2020
Year: 2020

Background: People with severe speech and motor impairment (SSMI) often uses a technique called eye pointing to communicate with outside world. One of their parents, caretakers or teachers hold a printed board in front of them and by analyzing their eye gaze manually, their intentions are interpreted. This technique is often error prone and time consuming and depends on a single caretaker. Objective: We aimed to automate the eye tracking process electronically by using commercially available tablet, computer or laptop and without requiring any dedicated hardware for eye gaze tracking. The eye gaze tracker is used to develop a video see through based AR (augmented reality) display that controls a robotic device with eye gaze and deployed for a fabric printing task. Methodology: We undertook a user centred design process and separately evaluated the web cam based gaze tracker and the video see through based human robot interaction involving users with SSMI. We also reported a user study on manipulating a robotic arm with webcam based eye gaze tracker. Results: Using our bespoke eye gaze controlled interface, able bodied users can select one of nine regions of screen at a median of less than 2 secs and users with SSMI can do so at a median of 4 secs. Using the eye gaze controlled human-robot AR display, users with SSMI could undertake representative pick and drop task at an average duration less than 15 secs and reach a randomly designated target within 60 secs using a COTS eye tracker and at an average time of 2 mins using the webcam based eye gaze tracker.
Pointwise Paraphrase Appraisal is Potentially Problematic
Venue: arXiv cs.CL 2020
Year: 2020

The prevailing approach for training and evaluating paraphrase identification models is constructed as a binary classification problem: the model is given a pair of sentences, and is judged by how accurately it classifies pairs as either paraphrases or non-paraphrases. This pointwise-based evaluation method does not match well the objective of most real world applications, so the goal of our work is to understand how models which perform well under pointwise evaluation may fail in practice and find better methods for evaluating paraphrase identification models. As a first step towards that goal, we show that although the standard way of fine-tuning BERT for paraphrase identification by pairing two sentences as one sequence results in a model with state-of-the-art performance, that model may perform poorly on simple tasks like identifying pairs with two identical sentences. Moreover, we show that these models may even predict a pair of randomly-selected sentences with higher paraphrase score than a pair of identical ones.
Interaction-Aware Trajectory Prediction of Connected Vehicles using CNN-LSTM Networks
Venue: arXiv cs.RO 2020
Year: 2020

Predicting the future trajectory of a surrounding vehicle in congested traffic is one of the basic abilities of an autonomous vehicle. In congestion, a vehicle's future movement is the result of its interaction with surrounding vehicles. A vehicle in congestion may have many neighbors in a relatively short distance, while only a small part of neighbors affect its future trajectory mostly. In this work, An interaction-aware method which predicts the future trajectory of an ego vehicle considering its interaction with eight surrounding vehicles is proposed. The dynamics of vehicles are encoded by LSTMs with shared weights, and the interaction is extracted with a simple CNN. The proposed model is trained and tested on trajectories extracted from the publicly accessible NGSIM US-101 dataset. Quantitative experimental results show that the proposed model outperforms previous models in terms of root-mean-square error (RMSE). Results visualization shows that the model is able to predict future trajectory induced by lane change before the vehicle operate obvious lateral movement to initiate lane changing.
Usage Analysis of Mobile Devices
Venue: arXiv cs.CY 2020
Year: 2020

Mobile devices have evolved from just communication devices into an indispensable part of people's lives in form of smartphones, tablets and smart watches. Devices are now more personal than ever and carry more information about a person than any other. Extracting user behaviour is rather difficult and time-consuming as most of the work previously has been manual or requires feature extraction. In this paper, a novel approach of user behavior detection is proposed with Deep Learning Network (DNN). Initial approach was to use recurrent neural network (RNN) along with LSTM for completely unsupervised analysis of mobile devices. Next approach is to extract features by using Long Short Term Memory (LSTM) to understand the user behaviour, which are then fed into the Convolution Neural Network (CNN). This work mainly concentrates on detection of user behaviour and anomaly detection for usage analysis of mobile devices. Both the approaches are compared against some baseline methods. Experiments are conducted on the publicly available dataset to show that these methods can successfully capture the user behaviors.
Adapting End-to-End Speech Recognition for Readable Subtitles
Venue: arXiv cs.CL 2020
Year: 2020

Automatic speech recognition (ASR) systems are primarily evaluated on transcription accuracy. However, in some use cases such as subtitling, verbatim transcription would reduce output readability given limited screen size and reading time. Therefore, this work focuses on ASR with output compression, a task challenging for supervised approaches due to the scarcity of training data. We first investigate a cascaded system, where an unsupervised compression model is used to post-edit the transcribed speech. We then compare several methods of end-to-end speech recognition under output length constraints. The experiments show that with limited data far less than needed for training a model from scratch, we can adapt a Transformer-based ASR model to incorporate both transcription and compression capabilities. Furthermore, the best performance in terms of WER and ROUGE scores is achieved by explicitly modeling the length constraints within the end-to-end ASR system.
Formal Methods with a Touch of Magic
Venue: arXiv cs.LO 2020
Year: 2020

Machine learning and formal methods have complimentary benefits and drawbacks. In this work, we address the controller-design problem with a combination of techniques from both fields. The use of black-box neural networks in deep reinforcement learning (deep RL) poses a challenge for such a combination. Instead of reasoning formally about the output of deep RL, which we call the {\em wizard}, we extract from it a decision-tree based model, which we refer to as the {\em magic book}. Using the extracted model as an intermediary, we are able to handle problems that are infeasible for either deep RL or formal methods by themselves. First, we suggest, for the first time, combining a magic book in a synthesis procedure. We synthesize a stand-alone correct-by-design controller that enjoys the favorable performance of RL. Second, we incorporate a magic book in a bounded model checking (BMC) procedure. BMC allows us to find numerous traces of the plant under the control of the wizard, which a user can use to increase the trustworthiness of the wizard and direct further training.
Incremental Real-Time Personalization in Human Activity Recognition Using Domain Adaptive Batch Normalization
Venue: arXiv cs.LG 2020
Year: 2020

Human Activity Recognition (HAR) from devices like smartphone accelerometers is a fundamental problem in ubiquitous computing. Machine learning based recognition models often perform poorly when applied to new users that were not part of the training data. Previous work has addressed this challenge by personalizing general recognition models to the unique motion pattern of a new user in a static batch setting. They require target user data to be available upfront. The more challenging online setting has received less attention. No samples from the target user are available in advance, but they arrive sequentially. Additionally, the motion pattern of users may change over time. Thus, adapting to new and forgetting old information must be traded off. Finally, the target user should not have to do any work to use the recognition system by, say, labeling any activities. Our work addresses all of these challenges by proposing an unsupervised online domain adaptation algorithm. Both classification and personalization happen continuously and incrementally in real time. Our solution works by aligning the feature distributions of all subjects, be they sources or the target, in hidden neural network layers. To this end, we normalize the input of a layer with user-specific mean and variance statistics. During training, these statistics are computed over user-specific batches. In the online phase, they are estimated incrementally for any new target user.
Artificial Intelligence (AI) and IT identity: Antecedents Identifying with AI Applications
Venue: arXiv cs.CY 2020
Year: 2020

In the age of Artificial Intelligence and automation, machines have taken over many key managerial tasks. Replacing managers with AI systems may have a negative impact on workers outcomes. It is unclear if workers receive the same benefits from their relationships with AI systems, raising the question: What degree does the relationship between AI systems and workers impact worker outcomes? We draw on IT identity to understand the influence of identification with AI systems on job performance. From this theoretical perspective, we propose a research model and conduct a survey of 97 MTurk workers to test the model. The findings reveal that work role identity and organizational identity are key determinants of identification with AI systems. Furthermore, the findings show that identification with AI systems does increase job performance.
Secure and User-Friendly Over-the-Air Firmware Distribution in a Portable Faraday Cage
Venue: arXiv cs.CR 2020
Year: 2020

Setting up a large-scale wireless sensor network is challenging, as firmware must be distributed and trust between sensor nodes and a backend needs to be established. To perform this task efficiently, we propose an approach named Box, which utilizes an intelligent Faraday cage (FC). The FC acquires firmware images and secret keys from a backend, patches the firmware with the keys and deploys those customized images over the air to sensor nodes placed in the FC. Electromagnetic shielding protects this exchange against passive attackers. We place few demands on the sensor node, not requiring additional hardware components or firmware customized by the manufacturer. We describe this novel workflow, implement the Box and a backend system and demonstrate the feasibility of our approach by batch-deploying firmware to multiple commercial off-the-shelf sensor nodes. We conduct a user-study with 31 participants with diverse backgrounds and find, that our approach is both faster and more user-friendly than firmware distribution over a wired connection.
Artificial Intelligence for Global Health: Learning From a Decade of Digital Transformation in Health Care
Venue: arXiv cs.CY 2020
Year: 2020

The health needs of those living in resource-limited settings are a vastly overlooked and understudied area in the intersection of machine learning (ML) and health care. While the use of ML in health care is more recently popularized over the last few years from the advancement of deep learning, low-and-middle income countries (LMICs) have already been undergoing a digital transformation of their own in health care over the last decade, leapfrogging milestones due to the adoption of mobile health (mHealth). With the introduction of new technologies, it is common to start afresh with a top-down approach, and implement these technologies in isolation, leading to lack of use and a waste of resources. In this paper, we outline the necessary considerations both from the perspective of current gaps in research, as well as from the lived experiences of health care professionals in resource-limited settings. We also outline briefly several key components of successful implementation and deployment of technologies within health systems in LMICs, including technical and cultural considerations in the development process relevant to the building of machine learning solutions. We then draw on these experiences to address where key opportunities for impact exist in resource-limited settings, and where AI/ML can provide the most benefit.
iTV: Inferring Traffic Violation-Prone Locations with Vehicle Trajectories and Road Environment Data
Venue: arXiv cs.CY 2020
Year: 2020

Traffic violations like illegal parking, illegal turning, and speeding have become one of the greatest challenges in urban transportation systems, bringing potential risks of traffic congestions, vehicle accidents, and parking difficulties. To maximize the utility and effectiveness of the traffic enforcement strategies aiming at reducing traffic violations, it is essential for urban authorities to infer the traffic violation-prone locations in the city. Therefore, we propose a low-cost, comprehensive, and dynamic framework to infer traffic violation-prone locations in cities based on the large-scale vehicle trajectory data and road environment data. Firstly, we normalize the trajectory data by map matching algorithms and extract key driving behaviors, i.e., turning behaviors, parking behaviors, and speeds of vehicles. Secondly, we restore spatiotemporal contexts of driving behaviors to get corresponding traffic restrictions such as no parking, no turning, and speed restrictions. After matching the traffic restrictions with driving behaviors, we get the traffic violation distribution. Finally, we extract the spatiotemporal patterns of traffic violations, and build a visualization system to showcase the inferred traffic violation-prone locations. To evaluate the effectiveness of the proposed method, we conduct extensive studies on large-scale, real-world vehicle GPS trajectories collected from two Chinese cities, respectively. Evaluation results confirm that the proposed framework infers traffic violation-prone locations effectively and efficiently, providing comprehensive decision supports for traffic enforcement strategies.