OpenRaiser/Intern-Atlas huggingface dataset:
Improving Sound Event Detection In Domestic Environments Using Sound Separation
Venue: arXiv cs.SD 2020
Year: 2020

Performing sound event detection on real-world recordings often implies dealing with overlapping target sound events and non-target sounds, also referred to as interference or noise. Until now these problems were mainly tackled at the classifier level. We propose to use sound separation as a pre-processing for sound event detection. In this paper we start from a sound separation model trained on the Free Universal Sound Separation dataset and the DCASE 2020 task 4 sound event detection baseline. We explore different methods to combine separated sound sources and the original mixture within the sound event detection. Furthermore, we investigate the impact of adapting the sound separation model to the sound event detection data on both the sound separation and the sound event detection.
Remix: Rebalanced Mixup
Venue: arXiv cs.CV 2020
Year: 2020

Deep image classifiers often perform poorly when training data are heavily class-imbalanced. In this work, we propose a new regularization technique, Remix, that relaxes Mixup's formulation and enables the mixing factors of features and labels to be disentangled. Specifically, when mixing two samples, while features are mixed in the same fashion as Mixup, Remix assigns the label in favor of the minority class by providing a disproportionately higher weight to the minority class. By doing so, the classifier learns to push the decision boundaries towards the majority classes and balance the generalization error between majority and minority classes. We have studied the state-of-the art regularization techniques such as Mixup, Manifold Mixup and CutMix under class-imbalanced regime, and shown that the proposed Remix significantly outperforms these state-of-the-arts and several re-weighting and re-sampling techniques, on the imbalanced datasets constructed by CIFAR-10, CIFAR-100, and CINIC-10. We have also evaluated Remix on a real-world large-scale imbalanced dataset, iNaturalist 2018. The experimental results confirmed that Remix provides consistent and significant improvements over the previous methods.
Distributed Training of Deep Learning Models: A Taxonomic Perspective
Venue: arXiv cs.DC 2020
Year: 2020

Distributed deep learning systems (DDLS) train deep neural network models by utilizing the distributed resources of a cluster. Developers of DDLS are required to make many decisions to process their particular workloads in their chosen environment efficiently. The advent of GPU-based deep learning, the ever-increasing size of datasets and deep neural network models, in combination with the bandwidth constraints that exist in cluster environments require developers of DDLS to be innovative in order to train high quality models quickly. Comparing DDLS side-by-side is difficult due to their extensive feature lists and architectural deviations. We aim to shine some light on the fundamental principles that are at work when training deep neural networks in a cluster of independent machines by analyzing the general properties associated with training deep learning models and how such workloads can be distributed in a cluster to achieve collaborative model training. Thereby we provide an overview of the different techniques that are used by contemporary DDLS and discuss their influence and implications on the training process. To conceptualize and compare DDLS, we group different techniques into categories, thus establishing a taxonomy of distributed deep learning systems.
Synthetic-to-Real Domain Adaptation for Lane Detection
Venue: arXiv cs.CV 2020
Year: 2020

Accurate lane detection, a crucial enabler for autonomous driving, currently relies on obtaining a large and diverse labeled training dataset. In this work, we explore learning from abundant, randomly generated synthetic data, together with unlabeled or partially labeled target domain data, instead. Randomly generated synthetic data has the advantage of controlled variability in the lane geometry and lighting, but it is limited in terms of photo-realism. This poses the challenge of adapting models learned on the unrealistic synthetic domain to real images. To this end we develop a novel autoencoder-based approach that uses synthetic labels unaligned with particular images for adapting to target domain data. In addition, we explore existing domain adaptation approaches, such as image translation and self-supervision, and adjust them to the lane detection task. We test all approaches in the unsupervised domain adaptation setting in which no target domain labels are available and in the semi-supervised setting in which a small portion of the target images are labeled. In extensive experiments using three different datasets, we demonstrate the possibility to save costly target domain labeling efforts. For example, using our proposed autoencoder approach on the llamas and tuSimple lane datasets, we can almost recover the fully supervised accuracy with only 10% of the labeled data. In addition, our autoencoder approach outperforms all other methods in the semi-supervised domain adaptation scenario.
Fair Colorful k-Center Clustering
Venue: arXiv cs.DS 2020
Year: 2020

An instance of colorful k-center consists of points in a metric space that are colored red or blue, along with an integer k and a coverage requirement for each color. The goal is to find the smallest radius \r{ho} such that there exist balls of radius \r{ho} around k of the points that meet the coverage requirements. The motivation behind this problem is twofold. First, from fairness considerations: each color/group should receive a similar service guarantee, and second, from the algorithmic challenges it poses: this problem combines the difficulties of clustering along with the subset-sum problem. In particular, we show that this combination results in strong integrality gap lower bounds for several natural linear programming relaxations. Our main result is an efficient approximation algorithm that overcomes these difficulties to achieve an approximation guarantee of 3, nearly matching the tight approximation guarantee of 2 for the classical k-center problem which this problem generalizes.
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Venue: arXiv cs.CY 2020
Year: 2020

This paper explores the important role of critical science, and in particular of post-colonial and decolonial theories, in understanding and shaping the ongoing advances in artificial intelligence. Artificial Intelligence (AI) is viewed as amongst the technological advances that will reshape modern societies and their relations. Whilst the design and deployment of systems that continually adapt holds the promise of far-reaching positive change, they simultaneously pose significant risks, especially to already vulnerable peoples. Values and power are central to this discussion. Decolonial theories use historical hindsight to explain patterns of power that shape our intellectual, political, economic, and social world. By embedding a decolonial critical approach within its technical practice, AI communities can develop foresight and tactics that can better align research and technology development with established ethical principles, centring vulnerable peoples who continue to bear the brunt of negative impacts of innovation and scientific progress. We highlight problematic applications that are instances of coloniality, and using a decolonial lens, submit three tactics that can form a decolonial field of artificial intelligence: creating a critical technical practice of AI, seeking reverse tutelage and reverse pedagogies, and the renewal of affective and political communities. The years ahead will usher in a wave of new scientific breakthroughs and technologies driven by AI research, making it incumbent upon AI communities to strengthen the social contract through ethical foresight and the multiplicity of intellectual perspectives available to us; ultimately supporting future technologies that enable greater well-being, with the goal of beneficence and justice for all.
Hyperparameter Optimization in Neural Networks via Structured Sparse Recovery
Venue: arXiv cs.LG 2020
Year: 2020

In this paper, we study two important problems in the automated design of neural networks -- Hyper-parameter Optimization (HPO), and Neural Architecture Search (NAS) -- through the lens of sparse recovery methods. In the first part of this paper, we establish a novel connection between HPO and structured sparse recovery. In particular, we show that a special encoding of the hyperparameter space enables a natural group-sparse recovery formulation, which when coupled with HyperBand (a multi-armed bandit strategy), leads to improvement over existing hyperparameter optimization methods. Experimental results on image datasets such as CIFAR-10 confirm the benefits of our approach. In the second part of this paper, we establish a connection between NAS and structured sparse recovery. Building upon ``one-shot'' approaches in NAS, we propose a novel algorithm that we call CoNAS by merging ideas from one-shot approaches with a techniques for learning low-degree sparse Boolean polynomials. We provide theoretical analysis on the number of validation error measurements. Finally, we validate our approach on several datasets and discover novel architectures hitherto unreported, achieving competitive (or better) results in both performance and search time compared to the existing NAS approaches.
Bespoke vs. Prêt-à-Porter Lottery Tickets: Exploiting Mask Similarity for Trainable Sub-Network Finding
Venue: arXiv cs.LG 2020
Year: 2020

The observation of sparse trainable sub-networks within over-parametrized networks - also known as Lottery Tickets (LTs) - has prompted inquiries around their trainability, scaling, uniqueness, and generalization properties. Across 28 combinations of image classification tasks and architectures, we discover differences in the connectivity structure of LTs found through different iterative pruning techniques, thus disproving their uniqueness and connecting emergent mask structure to the choice of pruning. In addition, we propose a consensus-based method for generating refined lottery tickets. This lottery ticket denoising procedure, based on the principle that parameters that always go unpruned across different tasks more reliably identify important sub-networks, is capable of selecting a meaningful portion of the architecture in an embarrassingly parallel way, while quickly discarding extra parameters without the need for further pruning iterations. We successfully train these sub-networks to performance comparable to that of ordinary lottery tickets.
A Survey of Real-Time Social-Based Traffic Detection
Venue: arXiv cs.SI 2020
Year: 2020

Online traffic news web sites do not always announce traffic events in areas in real-time. There is a capability to employ text mining and machine learning techniques on the twitter stream to perform event detection, in order to develop a real-time traffic detection system. In this present survey paper, we will deliberate the current state-of-art techniques in detecting traffic events in real-time focusing on five papers [1, 2, 3, 4, 5]. Lastly, applying text mining techniques and SVM classifiers in paper [2] gave the best results (i.e. 95.75% accuracy and 95.8% F1-score).
Giving Up Privacy For Security: A Survey On Privacy Trade-off During Pandemic Emergency
Venue: arXiv cs.CY 2020
Year: 2020

While the COVID-19 pandemic continues to be as complex as ever, the collection and exchange of data in the light of fighting coronavirus poses a major challenge for privacy systems around the globe. The disease's size and magnitude is not uncommon but it appears to be at the point of hysteria surrounding it. Consequently, in a very short time, extreme measures for dealing with the situation appear to have become the norm. Any such actions affect the privacy of individuals in particular. For some cases, there is intensive monitoring of the whole population while the medical data of those diagnosed with the virus is commonly circulated through institutions and nations. This may well be in the interest of saving the world from a deadly disease, but is it really appropriate and right? Although creative solutions have been implemented in many countries to address the issue, proponents of privacy are concerned that technologies will eventually erode privacy, while regulators and privacy supporters are worried about what kind of impact this could bring. While that tension has always been present, privacy has been thrown into sharp relief by the sheer urgency of containing an exponentially spreading virus. The essence of this dilemma indicates that establishing the right equilibrium will be the best solution. The jurisprudence concerning cases regarding the willingness of public officials to interfere with the constitutional right to privacy in the interests of national security or public health has repeatedly proven that a reasonable balance can be reached.
Automated Multi-Architectural Discovery of CFI-Resistant Code Gadgets
Venue: arXiv cs.CR 2020
Year: 2020

Memory corruption vulnerabilities are still a severe threat for software systems. To thwart the exploitation of such vulnerabilities, many different kinds of defenses have been proposed in the past. Most prominently, Control-Flow Integrity (CFI) has received a lot of attention recently. Several proposals were published that apply coarse-grained policies with a low performance overhead. However, their security remains questionable as recent attacks have shown. To ease the assessment of a given CFI implementation, we introduce a framework to discover code gadgets for code-reuse attacks that conform to coarse-grained CFI policies. For this purpose, binary code is extracted and transformed to a symbolic representation in an architecture-independent manner. Additionally, code gadgets are verified to provide the needed functionality for a security researcher. We show that our framework finds more CFI-compatible gadgets compared to other code gadget discovery tools. Furthermore, we demonstrate that code gadgets needed to bypass CFI solutions on the ARM architecture can be discovered by our framework as well.
Energy Optimization in Ultra-Dense Radio Access Networks via Traffic-Aware Cell Switching
Venue: arXiv cs.NI 2020
Year: 2020

Ultra-dense deployments in 5G, the next generation of cellular networks, are an alternative to provide ultra-high throughput by bringing the users closer to the base stations. On the other hand, 5G deployments must not incur a large increase in energy consumption in order to keep them cost-effective and most importantly to reduce the carbon footprint of cellular networks. We propose a reinforcement learning cell switching algorithm, to minimize the energy consumption in ultra-dense deployments without compromising the quality of service (QoS) experienced by the users. In this regard, the proposed algorithm can intelligently learn which small cells (SCs) to turn off at any given time based on the traffic load of the SCs and the macro cell. To validate the idea, we used the open call detail record (CDR) data set from the city of Milan, Italy, and tested our algorithm against typical operational benchmark solutions. With the obtained results, we demonstrate exactly when and how the proposed algorithm can provide energy savings, and moreover how this happens without reducing QoS of users. Most importantly, we show that our solution has a very similar performance to the exhaustive search, with the advantage of being scalable and less complex.
Cloud Based Big Data DNS Analytics at Turknet
Venue: arXiv cs.DC 2020
Year: 2020

Domain Name System (DNS) is a hierarchical distributed naming system for computers, services, or any resource connected to the Internet. A DNS resolves queries for URLs into IP addresses for the purpose of locating computer services and devices worldwide. As of now, analytical applications with a vast amount of DNS data are a challenging problem. Clustering the features of domain traffic from a DNS data has given necessity to the need for more sophisticated analytics platforms and tools because of the sensitivity of the data characterization. In this study, a cloud based big data application, based on Apache Spark, on DNS data is proposed, as well as a periodic trend pattern based on traffic to partition numerous domain names and region into separate groups by the characteristics of their query traffic time series. Preliminary experimental results on a Turknet DNS data in daily operations are discussed with business intelligence applications.
AutoLR: An Evolutionary Approach to Learning Rate Policies
Venue: arXiv cs.NE 2020
Year: 2020

The choice of a proper learning rate is paramount for good Artificial Neural Network training and performance. In the past, one had to rely on experience and trial-and-error to find an adequate learning rate. Presently, a plethora of state of the art automatic methods exist that make the search for a good learning rate easier. While these techniques are effective and have yielded good results over the years, they are general solutions. This means the optimization of learning rate for specific network topologies remains largely unexplored. This work presents AutoLR, a framework that evolves Learning Rate Schedulers for a specific Neural Network Architecture using Structured Grammatical Evolution. The system was used to evolve learning rate policies that were compared with a commonly used baseline value for learning rate. Results show that training performed using certain evolved policies is more efficient than the established baseline and suggest that this approach is a viable means of improving a neural network's performance.
A Benchmark of Medical Out of Distribution Detection
Venue: arXiv cs.LG 2020
Year: 2020

Motivation: Deep learning models deployed for use on medical tasks can be equipped with Out-of-Distribution Detection (OoDD) methods in order to avoid erroneous predictions. However it is unclear which OoDD method should be used in practice. Specific Problem: Systems trained for one particular domain of images cannot be expected to perform accurately on images of a different domain. These images should be flagged by an OoDD method prior to diagnosis. Our approach: This paper defines 3 categories of OoD examples and benchmarks popular OoDD methods in three domains of medical imaging: chest X-ray, fundus imaging, and histology slides. Results: Our experiments show that despite methods yielding good results on some categories of out-of-distribution samples, they fail to recognize images close to the training distribution. Conclusion: We find a simple binary classifier on the feature representation has the best accuracy and AUPRC on average. Users of diagnostic tools which employ these OoDD methods should still remain vigilant that images very close to the training distribution yet not in it could yield unexpected results.
Understanding the impact of the alphabetical ordering of names in user interfaces: a gender bias analysis
Venue: arXiv cs.HC 2020
Year: 2020

Listing people alphabetically on an electronic output device is a traditional technique, since alphabetical order is easily perceived by users and facilitates access to information. However, this apparently harmless technique, especially when the list is ordered by first name, needs to be used with caution by designers and programmers. We show, via empirical data analysis, that when an interface displays people's first name in alphabetical order in several pages/screens, each page/screen may have imbalances in respect to gender of its Top-k individuals.k represents the size of the list of names visualized first, which may be the number of names that fits in a screen page of a certain device.The research work was carried out with the analysis of actual datasets of names of five different countries. Each dataset has a person name and the frequency of adoption of the name in the country.Our analysis shows that, even though all countries have exhibit imbalance problems, the samples of individuals with Brazilian and Spanish first names are more prone to gender imbalance among their Top-k individuals. These results can be useful for designers and engineers to construct information systems that avoid gender bias induction.
Words as Art Materials: Generating Paintings with Sequential GANs
Venue: arXiv cs.CV 2020
Year: 2020

Converting text descriptions into images using Generative Adversarial Networks has become a popular research area. Visually appealing images have been generated successfully in recent years. Inspired by these studies, we investigated the generation of artistic images on a large variance dataset. This dataset includes images with variations, for example, in shape, color, and content. These variations in images provide originality which is an important factor for artistic essence. One major characteristic of our work is that we used keywords as image descriptions, instead of sentences. As the network architecture, we proposed a sequential Generative Adversarial Network model. The first stage of this sequential model processes the word vectors and creates a base image whereas the next stages focus on creating high-resolution artistic-style images without working on word vectors. To deal with the unstable nature of GANs, we proposed a mixture of techniques like Wasserstein loss, spectral normalization, and minibatch discrimination. Ultimately, we were able to generate painting images, which have a variety of styles. We evaluated our results by using the Fréchet Inception Distance score and conducted a user study with 186 participants.
Multi-Swarm Herding: Protecting against Adversarial Swarms
Venue: arXiv cs.MA 2020
Year: 2020

This paper studies a defense approach against one or more swarms of adversarial agents. In our earlier work, we employ a closed formation (`StringNet') of defending agents (defenders) around a swarm of adversarial agents (attackers) to confine their motion within given bounds, and guide them to a safe area. The control design relies on the assumption that the adversarial agents remain close enough to each other, i.e., within a prescribed connectivity region. To handle situations when the attackers no longer stay within such a connectivity region, but rather split into smaller swarms (clusters) to maximize the chance or impact of attack, this paper proposes an approach to learn the attacking sub-swarms and reassign defenders towards the attackers. We use a `Density-based Spatial Clustering of Application with Noise (DBSCAN)' algorithm to identify the spatially distributed swarms of the attackers. Then, the defenders are assigned to each identified swarm of attackers by solving a constrained generalized assignment problem. Simulations are provided to demonstrate the effectiveness of the approach.
T-REx: Table Repair Explanations
Venue: arXiv cs.DB 2020
Year: 2020

Data repair is a common and crucial step in many frameworks today, as applications may use data from different sources and of different levels of credibility. Thus, this step has been the focus of many works, proposing diverse approaches. To assist users in understanding the output of such data repair algorithms, we propose T-REx, a system for providing data repair explanations through Shapley values. The system is generic and not specific to a given repair algorithm or approach: it treats the algorithm as a black box. Given a specific table cell selected by the user, T-REx employs Shapley values to explain the significance of each constraint and each table cell in the repair of the cell of interest. T-REx then ranks the constraints and table cells according to their importance in the repair of this cell. This explanation allows users to understand the repair process, as well as to act based on this knowledge, to modify the most influencing constraints or the original database.