OpenRaiser/Intern-Atlas huggingface dataset:
A Framework for Auditing Data Center Energy Usage and Mitigating Environmental Footprint
Venue: arXiv cs.DC 2021
Year: 2021

As the Data Science field continues to mature, and we collect more data, the demand to store and analyze them will continue to increase. This increase in data availability and demand for analytics will put a strain on data centers and compute clusters-with implications for both energy costs and emissions. As the world battles a climate crisis, it is prudent for organizations with data centers to have a framework for combating increasing energy costs and emissions to meet demand for analytics work. In this paper, I present a generalized framework for organizations to audit data centers energy efficiency to understand the resources required to operate a given data center and effective steps organizations can take to improve data center efficiency and lower the environmental impact.
Free-space optical neural network based on thermal atomic nonlinearity
Venue: arXiv cs.ET 2021
Year: 2021

As artificial neural networks (ANNs) continue to make strides in wide-ranging and diverse fields of technology, the search for more efficient hardware implementations beyond conventional electronics is gaining traction. In particular, optical implementations potentially offer extraordinary gains in terms of speed and reduced energy consumption due to intrinsic parallelism of free-space optics. At the same time, a physical nonlinearity, a crucial ingredient of an ANN, is not easy to realize in free-space optics, which restricts the potential of this platform. This problem is further exacerbated by the need to perform the nonlinear activation also in parallel for each data point to preserve the benefit of linear free-space optics. Here, we present a free-space optical ANN with diffraction-based linear weight summation and nonlinear activation enabled by the saturable absorption of thermal atoms. We demonstrate, via both simulation and experiment, image classification of handwritten digits using only a single layer and observed 6-percent improvement in classification accuracy due to the optical nonlinearity compared to a linear model. Our platform preserves the massive parallelism of free-space optics even with physical nonlinearity, and thus opens the way for novel designs and wider deployment of optical ANNs.
VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference
Venue: arXiv cs.LG 2021
Year: 2021

Quantization enables efficient acceleration of deep neural networks by reducing model memory footprint and exploiting low-cost integer math hardware units. Quantization maps floating-point weights and activations in a trained model to low-bitwidth integer values using scale factors. Excessive quantization, reducing precision too aggressively, results in accuracy degradation. When scale factors are shared at a coarse granularity across many dimensions of each tensor, effective precision of individual elements within the tensor are limited. To reduce quantization-related accuracy loss, we propose using a separate scale factor for each small vector of ($\approx$16-64) elements within a single dimension of a tensor. To achieve an efficient hardware implementation, the per-vector scale factors can be implemented with low-bitwidth integers when calibrated using a two-level quantization scheme. We find that per-vector scaling consistently achieves better inference accuracy at low precision compared to conventional scaling techniques for popular neural networks without requiring retraining. We also modify a deep learning accelerator hardware design to study the area and energy overheads of per-vector scaling support. Our evaluation demonstrates that per-vector scaled quantization with 4-bit weights and activations achieves 37% area saving and 24% energy saving while maintaining over 75% accuracy for ResNet50 on ImageNet. 4-bit weights and 8-bit activations achieve near-full-precision accuracy for both BERT-base and BERT-large on SQuAD while reducing area by 26% compared to an 8-bit baseline.
A modular framework for extreme weather generation
Venue: arXiv cs.LG 2021
Year: 2021

Extreme weather events have an enormous impact on society and are expected to become more frequent and severe with climate change. In this context, resilience planning becomes crucial for risk mitigation and coping with these extreme events. Machine learning techniques can play a critical role in resilience planning through the generation of realistic extreme weather event scenarios that can be used to evaluate possible mitigation actions. This paper proposes a modular framework that relies on interchangeable components to produce extreme weather event scenarios. We discuss possible alternatives for each of the components and show initial results comparing two approaches on the task of generating precipitation scenarios.
Superfast Coloring in CONGEST via Efficient Color Sampling
Venue: arXiv cs.DC 2021
Year: 2021

We present a procedure for efficiently sampling colors in the {\congest} model. It allows nodes whose number of colors exceeds their number of neighbors by a constant fraction to sample up to $Θ(\log n)$ semi-random colors unused by their neighbors in $O(1)$ rounds, even in the distance-2 setting. This yields algorithms with $O(\log^* Δ)$ complexity for different edge-coloring, vertex coloring, and distance-2 coloring problems, matching the best possible. In particular, we obtain an $O(\log^* Δ)$-round CONGEST algorithm for $(1+ε)Δ$-edge coloring when $Δ\ge \log^{1+1/\log^*n} n$, and a poly($\log\log n$)-round algorithm for $(2Δ-1)$-edge coloring in general. The sampling procedure is inspired by a seminal result of Newman in communication complexity.
A Ranking Approach to Fair Classification
Venue: arXiv cs.LG 2021
Year: 2021

Algorithmic decision systems are increasingly used in areas such as hiring, school admission, or loan approval. Typically, these systems rely on labeled data for training a classification model. However, in many scenarios, ground-truth labels are unavailable, and instead we have only access to imperfect labels as the result of (potentially biased) human-made decisions. Despite being imperfect, historical decisions often contain some useful information on the unobserved true labels. In this paper, we focus on scenarios where only imperfect labels are available and propose a new fair ranking-based decision system based on monotonic relationships between legitimate features and the outcome. Our approach is both intuitive and easy to implement, and thus particularly suitable for adoption in real-world settings. More in detail, we introduce a distance-based decision criterion, which incorporates useful information from historical decisions and accounts for unwanted correlation between protected and legitimate features. Through extensive experiments on synthetic and real-world data, we show that our method is fair in the sense that a) it assigns the desirable outcome to the most qualified individuals, and b) it removes the effect of stereotypes in decision-making, thereby outperforming traditional classification algorithms. Additionally, we are able to show theoretically that our method is consistent with a prominent concept of individual fairness which states that "similar individuals should be treated similarly."
NELA-GT-2020: A Large Multi-Labelled News Dataset for The Study of Misinformation in News Articles
Venue: arXiv cs.CY 2021
Year: 2021

In this paper, we present an updated version of the NELA-GT-2019 dataset, entitled NELA-GT-2020. NELA-GT-2020 contains nearly 1.8M news articles from 519 sources collected between January 1st, 2020 and December 31st, 2020. Just as with NELA-GT-2018 and NELA-GT-2019, these sources come from a wide range of mainstream news sources and alternative news sources. Included in the dataset are source-level ground truth labels from Media Bias/Fact Check (MBFC) covering multiple dimensions of veracity. Additionally, new in the 2020 dataset are the Tweets embedded in the collected news articles, adding an extra layer of information to the data. The NELA-GT-2020 dataset can be found at https://doi.org/10.7910/DVN/CHMUYZ.
Moderator Factors of Software Security and Performance Verification
Venue: arXiv cs.SE 2021
Year: 2021

Context: Security and performance (S&P) are critical non-functional requirements on software systems. Therefore, verification activities should be included in the development process to identify related defects and avoiding S&P failures after deployment. However, the state of the practice of S&P verification is unclear, challenging academia to offer solutions for real-world problems faced by the S&P verification practitioners. Thus, identifying factors moderating the S&P verification helps software development organizations improve the S&P verification, releasing software that meets security and performance requirements. Objective: To present moderator factors influencing S&P verification activities and actions to promote S&P moderator factors. Method: Multiple case study using qualitative analysis of observational data to identify S&P moderators factors. Literature Rapid Reviews with Snowballing to strengthen confidence in the identified S&P moderators factors. Practitioners Survey to classify the S&P moderator factors regarding their relevance. Results: Identification of eight S&P moderator factors regarding organizational awareness, crossfunctional team, S&P requirements, support tools, verification environment, verification methodology, verification planning, and reuse practices. The literature reviews allowed us to confirm the identified S&P moderator factors and identify a set of actions to promote each of them. A survey with 37 valid participants allowed us to classify the identified S&P moderators factors and their actions relevant to S&P verification activities. Conclusions: The S&P moderator factors can be considered key points in which software development organizations should invest to implement or improve S&P verification activities.
Rationally Inattentive Utility Maximization for Interpretable Deep Image Classification
Venue: arXiv cs.LG 2021
Year: 2021

Are deep convolutional neural networks (CNNs) for image classification explainable by utility maximization with information acquisition costs? We demonstrate that deep CNNs behave equivalently (in terms of necessary and sufficient conditions) to rationally inattentive utility maximizers, a generative model used extensively in economics for human decision making. Our claim is based by extensive experiments on 200 deep CNNs from 5 popular architectures. The parameters of our interpretable model are computed efficiently via convex feasibility algorithms. As an application, we show that our economics-based interpretable model can predict the classification performance of deep CNNs trained with arbitrary parameters with accuracy exceeding 94% . This eliminates the need to re-train the deep CNNs for image classification. The theoretical foundation of our approach lies in Bayesian revealed preference studied in micro-economics. All our results are on GitHub and completely reproducible.
Training Federated GANs with Theoretical Guarantees: A Universal Aggregation Approach
Venue: arXiv cs.LG 2021
Year: 2021

Recently, Generative Adversarial Networks (GANs) have demonstrated their potential in federated learning, i.e., learning a centralized model from data privately hosted by multiple sites. A federatedGAN jointly trains a centralized generator and multiple private discriminators hosted at different sites. A major theoretical challenge for the federated GAN is the heterogeneity of the local data distributions. Traditional approaches cannot guarantee to learn the target distribution, which isa mixture of the highly different local distributions. This paper tackles this theoretical challenge, and for the first time, provides a provably correct framework for federated GAN. We propose a new approach called Universal Aggregation, which simulates a centralized discriminator via carefully aggregating the mixture of all private discriminators. We prove that a generator trained with this simulated centralized discriminator can learn the desired target distribution. Through synthetic and real datasets, we show that our method can learn the mixture of largely different distributions where existing federated GAN methods fail.
Large-Scale Visual Search with Binary Distributed Graph at Alibaba
Venue: arXiv cs.IR 2021
Year: 2021

Graph-based approximate nearest neighbor search has attracted more and more attentions due to its online search advantages. Numbers of methods studying the enhancement of speed and recall have been put forward. However, few of them focus on the efficiency and scale of offline graph-construction. For a deployed visual search system with several billions of online images in total, building a billion-scale offline graph in hours is essential, which is almost unachievable by most existing methods. In this paper, we propose a novel algorithm called Binary Distributed Graph to solve this problem. Specifically, we combine binary codes with graph structure to speedup online and offline procedures, and achieve comparable performance with the ones in real-value based scenarios by recalling more binary candidates. Furthermore, the graph-construction is optimized to completely distributed implementation, which significantly accelerates the offline process and gets rid of the limitation of memory and disk within a single machine. Experimental comparisons on Alibaba Commodity Data Set (more than three billion images) show that the proposed method outperforms the state-of-the-art with respect to the online/offline trade-off.
Fair Peer-to-Peer Content Delivery via Blockchain
Venue: arXiv cs.CR 2021
Year: 2021

Peer-to-peer (p2p) content delivery is promising to provide benefits like cost-saving and scalable peak-demand handling in comparison with conventional content delivery networks (CDNs) and complement the decentralized storage networks such as Filecoin. However, reliable p2p delivery requires proper enforcement of delivery fairness, i.e., the deliverers should be rewarded according to their in-time delivery. Unfortunately, most existing studies on delivery fairness are based on non-cooperative game-theoretic assumptions that are arguably unrealistic in the ad-hoc p2p setting. We for the first time put forth the expressive yet still minimalist securities for p2p content delivery, and give two efficient solutions FairDownload and FairStream via the blockchain for p2p downloading and p2p streaming scenarios, respectively. Our designs not only guarantee delivery fairness to ensure deliverers be paid (nearly) proportional to his in-time delivery, but also ensure the content consumers and content providers to be fairly treated. The fairness of each party can be guaranteed when the other two parties collude to arbitrarily misbehave. Moreover, the systems are efficient in the sense of attaining asymptotically optimal on-chain costs and optimal deliverer communication. We implement the protocols to build the prototype systems atop the Ethereum Ropsten network. Extensive experiments done in LAN and WAN settings showcase their high practicality.
PyART: Python API Recommendation in Real-Time
Venue: arXiv cs.SE 2021
Year: 2021

API recommendation in real-time is challenging for dynamic languages like Python. Many existing API recommendation techniques are highly effective, but they mainly support static languages. A few Python IDEs provide API recommendation functionalities based on type inference and training on a large corpus of Python libraries and third-party libraries. As such, they may fail to recommend or make poor recommendations when type information is missing or target APIs are project-specific. In this paper, we propose a novel approach, PyART, to recommend APIs for Python programs in real-time. It features a light-weight analysis to derives so-called optimistic data-flow, which is neither sound nor complete, but simulates the local data-flow information humans can derive. It extracts three kinds of features: data-flow, token similarity, and token co-occurrence, in the context of the program point where a recommendation is solicited. A predictive model is trained on these features using the Random Forest algorithm. Evaluation on 8 popular Python projects demonstrates that PyART can provide effective API recommendations. When historic commits can be leveraged, which is the target scenario of a state-of-the-art tool ARIREC, our average top-1 accuracy is over 50% and average top-10 accuracy over 70%, outperforming APIREC and Intellicode (i.e., the recommendation component in Visual Studio) by 28.48%-39.05% for top-1 accuracy and 24.41%-30.49% for top-10 accuracy. In other applications such as when historic comments are not available and cross-project recommendation, PyART also shows better overall performance. The time to make a recommendation is less than a second on average, satisfying the real-time requirement.
Interrogating the Black Box: Transparency through Information-Seeking Dialogues
Venue: arXiv cs.AI 2021
Year: 2021

This paper is preoccupied with the following question: given a (possibly opaque) learning system, how can we understand whether its behaviour adheres to governance constraints? The answer can be quite simple: we just need to "ask" the system about it. We propose to construct an investigator agent to query a learning agent -- the suspect agent -- to investigate its adherence to a given ethical policy in the context of an information-seeking dialogue, modeled in formal argumentation settings. This formal dialogue framework is the main contribution of this paper. Through it, we break down compliance checking mechanisms into three modular components, each of which can be tailored to various needs in a vast amount of ways: an investigator agent, a suspect agent, and an acceptance protocol determining whether the responses of the suspect agent comply with the policy. This acceptance protocol presents a fundamentally different approach to aggregation: rather than using quantitative methods to deal with the non-determinism of a learning system, we leverage the use of argumentation semantics to investigate the notion of properties holding consistently. Overall, we argue that the introduced formal dialogue framework opens many avenues both in the area of compliance checking and in the analysis of properties of opaque systems.
Reverb: A Framework For Experience Replay
Venue: arXiv cs.LG 2021
Year: 2021

A central component of training in Reinforcement Learning (RL) is Experience: the data used for training. The mechanisms used to generate and consume this data have an important effect on the performance of RL algorithms. In this paper, we introduce Reverb: an efficient, extensible, and easy to use system designed specifically for experience replay in RL. Reverb is designed to work efficiently in distributed configurations with up to thousands of concurrent clients. The flexible API provides users with the tools to easily and accurately configure the replay buffer. It includes strategies for selecting and removing elements from the buffer, as well as options for controlling the ratio between sampled and inserted elements. This paper presents the core design of Reverb, gives examples of how it can be applied, and provides empirical results of Reverb's performance characteristics.
Encoding and Decoding Construction D' Lattices for Power-Constrained Communications
Venue: arXiv cs.IT 2021
Year: 2021

This paper focuses on the encoding and decoding of Construction D' coding lattices that can be used with shaping lattices for power-constrained channels. Two encoding methods and a decoding algorithm for Construction D' lattices are given. A design of quasi-cyclic low-density parity-check (QC-LDPC) codes to form Construction D' lattices is presented. This allows construction of nested lattice codes which are good for coding, good for shaping, and have low complexity encoding and decoding. Numerical results of using $E_8$, $BW_{16}$ and Leech lattices for shaping a Construction D' lattice indicate that the shaping gains 0.65 dB, 0.86 dB and 1.03 dB are preserved, respectively.
Fast discovery of multidimensional subsequences for robust trajectory classification
Venue: arXiv cs.LG 2021
Year: 2021

Trajectory classification tasks became more complex as large volumes of mobility data are being generated every day and enriched with new sources of information, such as social networks and IoT sensors. Fast classification algorithms are essential for discovering knowledge in trajectory data for real applications. In this work we propose a method for fast discovery of subtrajectories with the reduction of the search space and the optimization of the MASTERMovelets method, which has proven to be effective for discovering interpretable patterns in classification problems.
AI-based Blackbox Code Deobfuscation: Understand, Improve and Mitigate
Venue: arXiv cs.CR 2021
Year: 2021

Code obfuscation aims at protecting Intellectual Property and other secrets embedded into software from being retrieved. Recent works leverage advances in artificial intelligence with the hope of getting blackbox deobfuscators completely immune to standard (whitebox) protection mechanisms. While promising, this new field of AI-based blackbox deobfuscation is still in its infancy. In this article we deepen the state of AI-based blackbox deobfuscation in three key directions: understand the current state-of-the-art, improve over it and design dedicated protection mechanisms. In particular, we define a novel generic framework for AI-based blackbox deobfuscation encompassing prior work and highlighting key components; we are the first to point out that the search space underlying code deobfuscation is too unstable for simulation-based methods (e.g., Monte Carlo Tres Search used in prior work) and advocate the use of robust methods such as S-metaheuritics; we propose the new optimized AI-based blackbox deobfuscator Xyntia which significantly outperforms prior work in terms of success rate (especially with small time budget) while being completely immune to the most recent anti-analysis code obfuscation methods; and finally we propose two novel protections against AI-based blackbox deobfuscation, allowing to counter Xyntia's powerful attacks.
Diagnosis of COVID-19 and Non-COVID-19 Patients by Classifying Only a Single Cough Sound
Venue: arXiv cs.SD 2021
Year: 2021

In this study, we proposed a machine learning-based system to distinguish patients with COVID-19 from non-COVID-19 patients by analyzing only a single cough sound. Two different data sets were used, one accessible for the public and the other available on request. After combining the data sets, the features were obtained from the cough sounds using the mel-frequency cepstral coefficients (MFCCs) method, and then they were classified with seven different machine learning classifiers. To determine the optimum values of hyperparameters for MFCCs and classifiers, the leave-one-out cross-validation (LOO-CV) strategy was implemented. Based on the results, the k-nearest neighbors classifier based on the Euclidean distance (k-NN Euclidean) with the accuracy rate, sensitivity of COVID-19, sensitivity of non-COVID-19, F-measure, and area under the ROC curve (AUC) of 0.9833, 1.0000, 0.9720, 0.9799, and 0.9860, respectively, is more successful than other classifiers. Finally, the best and most effective features were determined for each classifier using the sequential forward selection (SFS) method. According to the results, the proposed system is excellent compared with similar studies in the literature and can be easily used in smartphones and facilitate the diagnosis of COVID-19 patients. In addition, since the used data set includes reflex and unconscious coughs, the results showed that conscious or unconscious coughing has no effect on the diagnosis of COVID-19 patients based on the cough sound.