OpenRaiser/Intern-Atlas huggingface dataset:
Lower Bounds for Caching with Delayed Hits
Venue: arXiv cs.DS 2020
Year: 2020

Caches are a fundamental component of latency-sensitive computer systems. Recent work of [ASWB20] has initiated the study of delayed hits: a phenomenon in caches that occurs when the latency between the cache and backing store is much larger than the time between new requests. We present two results for the delayed hits caching model. (1) Competitive ratio lower bound. We prove that the competitive ratio of the algorithm in [ASWB20], and more generally of any deterministic online algorithm for delayed hits, is at least Omega(kZ), where k is the cache size and Z is the delay parameter. (2) Antimonotonicity of the delayed hits latency. Antimonotonicity is a naturally desirable property of cache latency: having a cache hit instead of a cache miss should result in lower overall latency. We prove that the latency of the delayed hits model is not antimonotone by exhibiting a scenario where having a cache hit instead of a miss results in an increase in overall latency. We additionally present a modification of the delayed hits model that makes the latency antimonotone.
Exploring Model Robustness with Adaptive Networks and Improved Adversarial Training
Venue: arXiv cs.LG 2020
Year: 2020

Adversarial training has proven to be effective in hardening networks against adversarial examples. However, the gained robustness is limited by network capacity and number of training samples. Consequently, to build more robust models, it is common practice to train on widened networks with more parameters. To boost robustness, we propose a conditional normalization module to adapt networks when conditioned on input samples. Our adaptive networks, once adversarially trained, can outperform their non-adaptive counterparts on both clean validation accuracy and robustness. Our method is objective agnostic and consistently improves both the conventional adversarial training objective and the TRADES objective. Our adaptive networks also outperform larger widened non-adaptive architectures that have 1.5 times more parameters. We further introduce several practical ``tricks'' in adversarial training to improve robustness and empirically verify their efficiency.
DANR: Discrepancy-aware Network Regularization
Venue: arXiv cs.LG 2020
Year: 2020

Network regularization is an effective tool for incorporating structural prior knowledge to learn coherent models over networks, and has yielded provably accurate estimates in applications ranging from spatial economics to neuroimaging studies. Recently, there has been an increasing interest in extending network regularization to the spatio-temporal case to accommodate the evolution of networks. However, in both static and spatio-temporal cases, missing or corrupted edge weights can compromise the ability of network regularization to discover desired solutions. To address these gaps, we propose a novel approach---{\it discrepancy-aware network regularization} (DANR)---that is robust to inadequate regularizations and effectively captures model evolution and structural changes over spatio-temporal networks. We develop a distributed and scalable algorithm based on the alternating direction method of multipliers (ADMM) to solve the proposed problem with guaranteed convergence to global optimum solutions. Experimental results on both synthetic and real-world networks demonstrate that our approach achieves improved performance on various tasks, and enables interpretation of model changes in evolving networks.
A Virtual Obstacle Course within Diverse Sensory Environments
Venue: arXiv cs.HC 2020
Year: 2020

We developed a novel assessment platform with untethered virtual reality, 3-dimensional sounds, and pressure sensing floor mat to help assess the walking balance and negotiation of obstacles given diverse sensory load and/or cognitive load. The platform provides an immersive 3D city-like scene with anticipated/unanticipated virtual obstacles. Participants negotiate the obstacles with perturbations of: auditory load by spatial audio, cognitive load by a memory task, and visual flow by generated by avatars movements at various amounts and speeds. A VR headset displays the scenes while providing real-time position and orientation of the participant's head. A pressure-sensing walkway senses foot pressure and visualizes it in a heatmap. The system helps to assess walking balance via pressure dynamics per foot, success rate of crossing obstacles, available response time as well as head kinematics in response to obstacles and multitasking. Based on the assessment, specific balance training and fall prevention program can be prescribed.
Attribute-Induced Bias Eliminating for Transductive Zero-Shot Learning
Venue: arXiv cs.CV 2020
Year: 2020

Transductive Zero-shot learning (ZSL) targets to recognize the unseen categories by aligning the visual and semantic information in a joint embedding space. There exist four kinds of domain biases in Transductive ZSL, i.e., visual bias and semantic bias between two domains and two visual-semantic biases in respective seen and unseen domains, but existing work only focuses on the part of them, which leads to severe semantic ambiguity during the knowledge transfer. To solve the above problem, we propose a novel Attribute-Induced Bias Eliminating (AIBE) module for Transductive ZSL. Specifically, for the visual bias between two domains, the Mean-Teacher module is first leveraged to bridge the visual representation discrepancy between two domains with unsupervised learning and unlabelled images. Then, an attentional graph attribute embedding is proposed to reduce the semantic bias between seen and unseen categories, which utilizes the graph operation to capture the semantic relationship between categories. Besides, to reduce the semantic-visual bias in the seen domain, we align the visual center of each category, instead of the individual visual data point, with the corresponding semantic attributes, which further preserves the semantic relationship in the embedding space. Finally, for the semantic-visual bias in the unseen domain, an unseen semantic alignment constraint is designed to align visual and semantic space in an unsupervised manner. The evaluations on several benchmarks demonstrate the effectiveness of the proposed method, e.g., obtaining the 82.8%/75.5%, 97.1%/82.5%, and 73.2%/52.1% for Conventional/Generalized ZSL settings for CUB, AwA2, and SUN datasets, respectively.
A New Accelerated Stochastic Gradient Method with Momentum
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we propose a novel accelerated stochastic gradient method with momentum, which momentum is the weighted average of previous gradients. The weights decays inverse proportionally with the iteration times. Stochastic gradient descent with momentum (Sgdm) use weights that decays exponentially with the iteration times to generate an momentum term. Using exponentially decaying weights, variants of Sgdm with well designed and complicated formats have been proposed to achieve better performance. The momentum update rules of our method is as simple as that of Sgdm. We provide theoretical convergence properties analyses for our method, which show both the exponentially decay weights and our inverse proportionally decay weights can limit the variance of the moving direction of parameters to be optimized to a region. Experimental results empirically show that our method works well with practical problems and outperforms Sgdm, and it outperforms Adam in convolutional neural networks.
Modified Segmentation Algorithm for Recognition of Older Geez Scripts Written on Vellum
Venue: arXiv cs.CV 2020
Year: 2020

Recognition of handwritten document aims at transforming document images into a machine understandable format. Handwritten document recognition is the most challenging area in the field of pattern recognition. It becomes more complex when a document was written on vellum before hundreds of years, like older Geez scripts. In this study, we introduced a modified segmentation approach to recognize older Geez scripts. We used adaptive filtering for noise reduction, Isodata iterative global thresholding for document image binarization, modified bounding box projection to segment distinct strokes between Geez characters, numbers, and punctuation marks. SVM multiclass classifier scored 79.32% recognition accuracy with the modified segmentation algorithm.
Cloud-scale VM Deflation for Running Interactive Applications On Transient Servers
Venue: arXiv cs.DC 2020
Year: 2020

Transient computing has become popular in public cloud environments for running delay-insensitive batch and data processing applications at low cost. Since transient cloud servers can be revoked at any time by the cloud provider, they are considered unsuitable for running interactive application such as web services. In this paper, we present VM deflation as an alternative mechanism to server preemption for reclaiming resources from transient cloud servers under resource pressure. Using real traces from top-tier cloud providers, we show the feasibility of using VM deflation as a resource reclamation mechanism for interactive applications in public clouds. We show how current hypervisor mechanisms can be used to implement VM deflation and present cluster deflation policies for resource management of transient and on-demand cloud VMs. Experimental evaluation of our deflation system on a Linux cluster shows that microservice-based applications can be deflated by up to 50\% with negligible performance overhead. Our cluster-level deflation policies allow overcommitment levels as high as 50\%, with less than a 1\% decrease in application throughput, and can enable cloud platforms to increase revenue by 30\%.
Learning to Recognise Words using Visually Grounded Speech
Venue: arXiv cs.CL 2020
Year: 2020

We investigated word recognition in a Visually Grounded Speech model. The model has been trained on pairs of images and spoken captions to create visually grounded embeddings which can be used for speech to image retrieval and vice versa. We investigate whether such a model can be used to recognise words by embedding isolated words and using them to retrieve images of their visual referents. We investigate the time-course of word recognition using a gating paradigm and perform a statistical analysis to see whether well known word competition effects in human speech processing influence word recognition. Our experiments show that the model is able to recognise words, and the gating paradigm reveals that words can be recognised from partial input as well and that recognition is negatively influenced by word competition from the word initial cohort.
A review of information security aspects of the emerging COVID-19 contact tracing mobile phone applications
Venue: arXiv cs.CR 2020
Year: 2020

This paper discusses the aspects of data reliability and user privacy for the emerging practice of mobile phone based contact tracing for the COVID-19 pandemic. Various countries and large technology companies have already used or plan to design and use mobile phone based solutions, in an effort to urgently expedite the process of identifying people who may have been exposed to the disease and limit its spread to the general population. However, serious concerns have been raised both in terms of the validity of the collected data as well as the extent to which implemented approaches can breach the privacy of the mobile phone users. This review examines the weaknesses of existing implementations and concludes with specific recommendations that can contribute towards increasing the safety of infrastructures that collect and process this kind of information, as well as the adoption and acceptance of these solutions from the public.
How to extend the Single-Processor Paradigm to the Explicitly Many-Processor Approach
Venue: arXiv cs.AR 2020
Year: 2020

The computing paradigm invented for processing a small amount of data on a single segregated processor cannot meet the challenges set by the present-day computing demands. The paper proposes a new computing paradigm (extending the old one to use several processors explicitly) and discusses some questions of its possible implementation. Some advantages of the implemented approach, illustrated with the results of a loosely-timed simulator, are presented.
Quantization Games on Social Networks and Language Evolution
Venue: arXiv cs.IT 2020
Year: 2020

We consider a strategic network quantizer design setting where agents must balance fidelity in representing their local source distributions against their ability to successfully communicate with other connected agents. We study the problem as a network game and show existence of Nash equilibrium quantizers. For any agent, under Nash equilibrium, the word representing a given partition region is the conditional expectation of the mixture of local and social source probability distributions within the region. Since having knowledge of the original source of information in the network may not be realistic, we show that under certain conditions, the agents need not know the source origin and yet still settle on a Nash equilibrium using only the observed sources. Further, the network may converge to equilibrium through a distributed version of the Lloyd-Max algorithm. In contrast to traditional results in the evolution of language, we find several vocabularies may coexist in the Nash equilibrium, with each individual having exactly one of these vocabularies. The overlap between vocabularies is high for individuals that communicate frequently and have similar local sources. Finally, we argue that error in translation along a chain of communication does not grow if and only if the chain consists of agents with shared vocabulary. Numerical results are given.
Learning Distributed Controllers for V-Formation
Venue: arXiv cs.MA 2020
Year: 2020

We show how a high-performing, fully distributed and symmetric neural V-formation controller can be synthesized from a Centralized MPC (Model Predictive Control) controller using Deep Learning. This result is significant as we also establish that under very reasonable conditions, it is impossible to achieve V-formation using a deterministic, distributed, and symmetric controller. The learning process we use for the neural V-formation controller is significantly enhanced by CEGkR, a Counterexample-Guided k-fold Retraining technique we introduce, which extends prior work in this direction in important ways. Our experimental results show that our neural V-formation controller generalizes to a significantly larger number of agents than for which it was trained (from 7 to 15), and exhibits substantial speedup over the MPC-based controller. We use a form of statistical model checking to compute confidence intervals for our neural V-formation controller's convergence rate and time to convergence.
F-IVM: Learning over Fast-Evolving Relational Data
Venue: arXiv cs.DB 2020
Year: 2020

F-IVM is a system for real-time analytics such as machine learning applications over training datasets defined by queries over fast-evolving relational databases. We will demonstrate F-IVM for three such applications: model selection, Chow-Liu trees, and ridge linear regression.
Transcription-Enriched Joint Embeddings for Spoken Descriptions of Images and Videos
Venue: arXiv cs.CV 2020
Year: 2020

In this work, we propose an effective approach for training unique embedding representations by combining three simultaneous modalities: image and spoken and textual narratives. The proposed methodology departs from a baseline system that spawns a embedding space trained with only spoken narratives and image cues. Our experiments on the EPIC-Kitchen and Places Audio Caption datasets show that introducing the human-generated textual transcriptions of the spoken narratives helps to the training procedure yielding to get better embedding representations. The triad speech, image and words allows for a better estimate of the point embedding and show an improving of the performance within tasks like image and speech retrieval, even when text third modality, text, is not present in the task.
Time Variable Minimum Torque Trajectory Optimization for Autonomous Excavator
Venue: arXiv cs.RO 2020
Year: 2020

In this paper, we present a minimal torque and time variable trajectory optimization method for autonomous excavator considering the soil-tool interaction. The method formulates the excavation motion generation as a trajectory optimization problem and takes into account geometric, kinematic and dynamics constraints. To generate time-efficient trajectory and improve the overall optimization efficiency, we propose a time variable trajectory optimization mechanism so that the time intervals between the keypoints along the trajectory subject to the optimization. As a result, the method uses few keypoints and reduces the total number of optimization variables. We further introduce a soil-tool interaction force model, which considers the geometric shape of the bucket and the physical properties of the soil. The experimental result on a high fidelity dynamic simulator shows our method can generate feasible trajectories, which satisfy excavation task constraints and are adaptive to different soil conditions.
Fog Computing for Smart Grids: Challenges and Solutions
Venue: arXiv cs.DC 2020
Year: 2020

Smart grids (SGs) enable integration of diverse power sources including renewable energy resources. They can contribute to the reduction of harmful gas emission, and support two-way information flow to enhance energy efficiency, along with small-scale Microgrids, acting as a promising solution to cope with environmental problems. However, with the emerging of mission-critical and delay-sensitive applications, traditional Cloud-based data processing mode becomes less satisfying. The use of Fog computing to empower the edge-side processing capability of Smart grid systems is considered as a potential solution to address the problem. In this chapter, we aim to offer a comprehensive analysis of application of Fog computing in Smart grids. We begin with an overview of Smart grids and Fog computing. Then, by surveying the existing research, we summarize the main Fog computing enabled Smart grid applications, key problems and the possible methods. We conclude the chapter by discussing the research challenges and future directions.
Efficient EUD Parsing
Venue: arXiv cs.CL 2020
Year: 2020

We present the system submission from the FASTPARSE team for the EUD Shared Task at IWPT 2020. We engaged with the task by focusing on efficiency. For this we considered training costs and inference efficiency. Our models are a combination of distilled neural dependency parsers and a rule-based system that projects UD trees into EUD graphs. We obtained an average ELAS of 74.04 for our official submission, ranking 4th overall.
Distilling Neural Networks for Greener and Faster Dependency Parsing
Venue: arXiv cs.CL 2020
Year: 2020

The carbon footprint of natural language processing research has been increasing in recent years due to its reliance on large and inefficient neural network implementations. Distillation is a network compression technique which attempts to impart knowledge from a large model to a smaller one. We use teacher-student distillation to improve the efficiency of the Biaffine dependency parser which obtains state-of-the-art performance with respect to accuracy and parsing speed (Dozat and Manning, 2017). When distilling to 20\% of the original model's trainable parameters, we only observe an average decrease of $\sim$1 point for both UAS and LAS across a number of diverse Universal Dependency treebanks while being 2.30x (1.19x) faster than the baseline model on CPU (GPU) at inference time. We also observe a small increase in performance when compressing to 80\% for some treebanks. Finally, through distillation we attain a parser which is not only faster but also more accurate than the fastest modern parser on the Penn Treebank.
Sarcasm Detection using Context Separators in Online Discourse
Venue: arXiv cs.CL 2020
Year: 2020

Sarcasm is an intricate form of speech, where meaning is conveyed implicitly. Being a convoluted form of expression, detecting sarcasm is an assiduous problem. The difficulty in recognition of sarcasm has many pitfalls, including misunderstandings in everyday communications, which leads us to an increasing focus on automated sarcasm detection. In the second edition of the Figurative Language Processing (FigLang 2020) workshop, the shared task of sarcasm detection released two datasets, containing responses along with their context sampled from Twitter and Reddit. In this work, we use RoBERTa_large to detect sarcasm in both the datasets. We further assert the importance of context in improving the performance of contextual word embedding based models by using three different types of inputs - Response-only, Context-Response, and Context-Response (Separated). We show that our proposed architecture performs competitively for both the datasets. We also show that the addition of a separation token between context and target response results in an improvement of 5.13% in the F1-score in the Reddit dataset.
Pruning via Iterative Ranking of Sensitivity Statistics
Venue: arXiv cs.LG 2020
Year: 2020

With the introduction of SNIP [arXiv:1810.02340v2], it has been demonstrated that modern neural networks can effectively be pruned before training. Yet, its sensitivity criterion has since been criticized for not propagating training signal properly or even disconnecting layers. As a remedy, GraSP [arXiv:2002.07376v1] was introduced, compromising on simplicity. However, in this work we show that by applying the sensitivity criterion iteratively in smaller steps - still before training - we can improve its performance without difficult implementation. As such, we introduce 'SNIP-it'. We then demonstrate how it can be applied for both structured and unstructured pruning, before and/or during training, therewith achieving state-of-the-art sparsity-performance trade-offs. That is, while already providing the computational benefits of pruning in the training process from the start. Furthermore, we evaluate our methods on robustness to overfitting, disconnection and adversarial attacks as well.
Implementing AI-powered semantic character recognition in motor racing sports
Venue: arXiv cs.CV 2020
Year: 2020

Oftentimes TV producers of motor-racing programs overlay visual and textual media to provide on-screen context about drivers, such as a driver's name, position or photo. Typically this is accomplished by a human producer who visually identifies the drivers on screen, manually toggling the contextual media associated to each one and coordinating with cameramen and other TV producers to keep the racer in the shot while the contextual media is on screen. This labor-intensive and highly dedicated process is mostly suited to static overlays and makes it difficult to overlay contextual information about many drivers at the same time in short shots. This paper presents a system that largely automates these tasks and enables dynamic overlays using deep learning to track the drivers as they move on screen, without human intervention. This system is not merely theoretical, but an implementation has already been deployed during live races by a TV production company at Formula E races. We present the challenges faced during the implementation and discuss the implications. Additionally, we cover future applications and roadmap of this new technological development.
Using competency questions to select optimal clustering structures for residential energy consumption patterns
Venue: arXiv cs.LG 2020
Year: 2020

During cluster analysis domain experts and visual analysis are frequently relied on to identify the optimal clustering structure. This process tends to be adhoc, subjective and difficult to reproduce. This work shows how competency questions can be used to formalise expert knowledge and application requirements for context specific evaluation of a clustering application in the residential energy consumption sector.