Reviewerly/exHarmony huggingface dataset:
Explainable Deep Learning: A Field Guide for the Uninitiated

Deep neural networks (DNNs) are an indispensable machine learning tool despite the difficulty of diagnosing what aspects of a model’s input drive its decisions. In countless real-world domains, from legislation and law enforcement to healthcare, such diagnosis is essential to ensure that DNN decisions are driven by aspects appropriate in the context of its use. The development of methods and studies enabling the explanation of a DNN’s decisions has thus blossomed into an active and broad area of research. The field’s complexity is exacerbated by competing definitions of what it means “to explain” the actions of a DNN and to evaluate an approach’s “ability to explain”. This article offers a field guide to explore the space of explainable deep learning for those in the AI/ML field who are uninitiated. The field guide: i) Introduces three simple dimensions defining the space of foundational methods that contribute to explainable deep learning, ii) discusses the evaluations for model explanations, iii) places explainability in the context of other related deep learning research areas, and iv) discusses user-oriented explanation design and future directions. We hope the guide is seen as a starting point for those embarking on this research field.
Models Matter, So Does Training: An Empirical Study of CNNs for Optical Flow Estimation

We investigate two crucial and closely-related aspects of CNNs for optical flow estimation: models and training. First, we design a compact but effective CNN model, called PWC-Net, according to simple and well-established principles: pyramidal processing, warping, and cost volume processing. PWC-Net is 17 times smaller in size, 2 times faster in inference, and 11 percent more accurate on Sintel final than the recent FlowNet2 model. It is the winning entry in the optical flow competition of the robust vision challenge. Next, we experimentally analyze the sources of our performance gains. In particular, we use the same training procedure for PWC-Net to retrain FlowNetC, a sub-network of FlowNet2. The retrained FlowNetC is 56 percent more accurate on Sintel final than the previously trained one and even 5 percent more accurate than the FlowNet2 model. We further improve the training procedure and increase the accuracy of PWC-Net on Sintel by 10 percent and on KITTI 2012 and 2015 by 20 percent. Our newly trained model parameters and training protocols are available on https://github.com/NVlabs/PWC-Net.
GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge

Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context.Traditional supervised methods rarely take into consideration the lexical resources like WordNet, which are widely utilized in knowledge-based methods.Recent studies have shown the effectiveness of incorporating gloss (sense definition) into neural networks for WSD.However, compared with traditional word expert supervised methods, they have not achieved much improvement.In this paper, we focus on how to better leverage gloss knowledge in a supervised neural WSD system.We construct context-gloss pairs and propose three BERT-based models for WSD.We fine-tune the pre-trained BERT model and achieve new state-of-the-art results on WSD task.
O2U-Net: A Simple Noisy Label Detection Approach for Deep Neural Networks

This paper proposes a novel noisy label detection approach, named O2U-net, for deep neural networks without human annotations. Different from prior work which requires specifically designed noise-robust loss functions or networks, O2U-net is easy to implement but effective. It only requires adjusting the hyper-parameters of the deep network to make its status transfer from overfitting to underfitting (O2U) cyclically. The losses of each sample are recorded during iterations. The higher the normalized average loss of a sample, the higher the probability of being noisy labels. O2U-net is naturally compatible with active learning and other human annotation approaches. This introduces extra flexibility for learning with noisy labels. We conduct sufficient experiments on multiple datasets in various settings. The experimental results prove the state-of-the-art of O2S-net.
Study of Variants of Extreme Learning Machine (ELM) Brands and its Performance Measure on Classification Algorithm

Recently, the feed-forward neural network is functioning with slow computation time and increased gain. The weight vector and biases in the neural network can be tuned based on performing intelligent assignment for simple generalized operation. This drawback of FFNN is solved by using various ELM algorithms based on the applications issues. ELM algorithms have redesigned the existing neural networks with network components such as hidden nodes, weights, and biases. The selection of hidden nodes is randomly determined and leverages good accuracy than conservative methods. The main aim of this research article is to explain variants of ELM advances for different applications. This procedure can be improved and optimized by using the neural network with novel feed-forward algorithm. The nodes will mainly perform due to the above factors, which are tuning for inverse operation. The ELM essence should be incorporated to reach a faster learning speed and less computation time with minimum human intervention. This research article consists of the real essence of ELM and a briefly explained algorithm for classification purpose. This research article provides clear information on the variants of ELM for different classification tasks. Finally, this research article has discussed the future extension of ELM for several applications based on the function approximation.
Identifying, categorizing and mitigating threats to validity in software engineering secondary studies

Secondary studies are vulnerable to threats to validity. Although, mitigating these threats is crucial for the credibility of these studies, we currently lack a systematic approach to identify, categorize and mitigate threats to validity for secondary studies. In this paper, we review the corpus of secondary studies, with the aim to identify: (a) the trend of reporting threats to validity, (b) the most common threats to validity and corresponding mitigation actions, and (c) possible categories in which threats to validity can be classified. To achieve this goal we employ the tertiary study research method that is used for synthesizing knowledge from existing secondary studies. In particular, we collected data from more than 100 studies, published until December 2016 in top quality software engineering venues (both journals and conference). Our results suggest that in recent years, secondary studies are more likely to report their threats to validity. However, the presentation of such threats is rather ad hoc, e.g., the same threat may be presented with a different name, or under a different category. To alleviate this problem, we propose a classification schema for reporting threats to validity and possible mitigation actions. Both the classification of threats and the associated mitigation actions have been validated by an empirical study, i.e., Delphi rounds with experts. Based on the proposed schema, we provide a checklist, which authors of secondary studies can use for identifying and categorizing threats to validity and corresponding mitigation actions, while readers of secondary studies can use the checklist for assessing the validity of the reported results.
Pan-Renal Cell Carcinoma classification and survival prediction from histopathology images using deep learning

Abstract Histopathological images contain morphological markers of disease progression that have diagnostic and predictive values. In this study, we demonstrate how deep learning framework can be used for an automatic classification of Renal Cell Carcinoma (RCC) subtypes, and for identification of features that predict survival outcome from digital histopathological images. Convolutional neural networks (CNN’s) trained on whole-slide images distinguish clear cell and chromophobe RCC from normal tissue with a classification accuracy of 93.39% and 87.34%, respectively. Further, a CNN trained to distinguish clear cell, chromophobe and papillary RCC achieves a classification accuracy of 94.07%. Here, we introduced a novel support vector machine-based method that helped to break the multi-class classification task into multiple binary classification tasks which not only improved the performance of the model but also helped to deal with data imbalance. Finally, we extracted the morphological features from high probability tumor regions identified by the CNN to predict patient survival outcome of most common clear cell RCC. The generated risk index based on both tumor shape and nuclei features are significantly associated with patient survival outcome. These results highlight that deep learning can play a role in both cancer diagnosis and prognosis.
Fast is better than free: Revisiting adversarial training

Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with $ε=8/255$ in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at $ε=2/255$ in 12 hours, in comparison to past work based on "free" adversarial training which took 10 and 50 hours to reach the same respective thresholds. Finally, we identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at https://github.com/locuslab/fast_adversarial.
Handheld multi-frame super-resolution

Compared to DSLR cameras, smartphone cameras have smaller sensors, which limits their spatial resolution; smaller apertures, which limits their light gathering ability; and smaller pixels, which reduces their signal-to-noise ratio. The use of color filter arrays (CFAs) requires demosaicing, which further degrades resolution. In this paper, we supplant the use of traditional demosaicing in single-frame and burst photography pipelines with a multiframe super-resolution algorithm that creates a complete RGB image directly from a burst of CFA raw images. We harness natural hand tremor, typical in handheld photography, to acquire a burst of raw frames with small offsets. These frames are then aligned and merged to form a single image with red, green, and blue values at every pixel site. This approach, which includes no explicit demosaicing step, serves to both increase image resolution and boost signal to noise ratio. Our algorithm is robust to challenging scene conditions: local motion, occlusion, or scene changes. It runs at 100 milliseconds per 12-megapixel RAW input burst frame on mass-produced mobile phones. Specifically, the algorithm is the basis of the Super-Res Zoom feature, as well as the default merge method in Night Sight mode (whether zooming or not) on Google's flagship phone.
Photon-Mediated Spin-Exchange Dynamics of Spin-1 Atoms

We report direct observations of photon-mediated spin-exchange interactions in an atomic ensemble. Interactions extending over a distance of 500 μm are generated within a cloud of cold rubidium atoms coupled to a single mode of light in an optical resonator. We characterize the system via quench dynamics and imaging of the local magnetization, verifying the coherence of the interactions and demonstrating optical control of their strength and sign. Furthermore, by initializing the spin-1 system in the m_{f}=0 Zeeman state, we observe correlated pair creation in the m_{f}=±1 states, a process analogous to spontaneous parametric down-conversion and to spin mixing in Bose-Einstein condensates. Our work opens new opportunities in quantum simulation with long-range interactions and in entanglement-enhanced metrology.
Physical secure optical communication based on private chaotic spectral phase encryption/decryption

We propose and demonstrate a novel physical, secure high-speed optical communication scheme based on synchronous chaotic spectral phase encryption (CSPE) and decryption (CSPD). The CSPE is performed by a module composed of two dispersion components and one phase modulator (PM) between them, and the CSPD is carried out by a twin module with reverse dispersions and inverse PM driving signals. The PM driving signals of the CSPE and CSPD modules are privately synchronized chaotic signals that are independently generated by local external-cavity semiconductor lasers subject to common injection. The numerical results indicate that with the CSPE, the original message can be encrypted as a noise-like signal, and the timing clock of the original message is efficiently hidden in the encrypted signal. Based on the private synchronization of the chaotic PM driving signals, only the legal receiver can decrypt the message correctly, while the eavesdropper is not able to intercept a useful message. Moreover, the proposed scheme can also support secure symmetric bidirectional high-speed WDM transmissions. This work shows a prospective way to implement high-speed secure optical communications at the physical layer.
A novel secure data transmission scheme in industrial internet of things

The industrial Internet of Things (IoT) is a trend of factory development and a basic condition of intelligent factory. It is very important to ensure the security of data transmission in industrial IoT. Applying a new chaotic secure communication scheme to address the security problem of data transmission is the main contribution of this paper. The scheme is proposed and studied based on the synchronization of different-structure fractional-order chaotic systems with different order. The Lyapunov stability theory is used to prove the synchronization between the fractional-order drive system and the response system. The encryption and decryption process of the main data signals is implemented by using the n-shift encryption principle. We calculate and analyze the key space of the scheme. Numerical simulations are introduced to show the effectiveness of theoretical approach we proposed.
UltraNest - a robust, general purpose Bayesian inference engine

UltraNest is a general-purpose Bayesian inference package for parameter estimation and model comparison.It allows fitting arbitrary models specified as likelihood functions written in Python, C, C++, Fortran, Julia or R. With a focus on correctness and speed (in that order), UltraNest is especially useful for multi-modal or non-Gaussian parameter spaces, computational expensive models, in robust pipelines.Parallelisation to computing clusters and resuming incomplete runs is available.
Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalization

Overparameterized neural networks can be highly accurate on average on an i.i.d. test set yet consistently fail on atypical groups of the data (e.g., by learning spurious correlations that hold on average but not in such groups). Distributionally robust optimization (DRO) allows us to learn models that instead minimize the worst-case training loss over a set of pre-defined groups. However, we find that naively applying group DRO to overparameterized neural networks fails: these models can perfectly fit the training data, and any model with vanishing average training loss also already has vanishing worst-case training loss. Instead, the poor worst-case performance arises from poor generalization on some groups. By coupling group DRO models with increased regularization---a stronger-than-typical L2 penalty or early stopping---we achieve substantially higher worst-group accuracies, with 10-40 percentage point improvements on a natural language inference task and two image tasks, while maintaining high average accuracies. Our results suggest that regularization is important for worst-group generalization in the overparameterized regime, even if it is not needed for average generalization. Finally, we introduce a stochastic optimization algorithm, with convergence guarantees, to efficiently train group DRO models.
BERTScore: Evaluating Text Generation with BERT

We propose BERTScore, an automatic evaluation metric for text generation. Analogously to common metrics, BERTScore computes a similarity score for each token in the candidate sentence with each token in the reference sentence. However, instead of exact matches, we compute token similarity using contextual embeddings. We evaluate using the outputs of 363 machine translation and image captioning systems. BERTScore correlates better with human judgments and provides stronger model selection performance than existing metrics. Finally, we use an adversarial paraphrase detection task to show that BERTScore is more robust to challenging examples when compared to existing metrics.
Convolutional neural networks for object detection in aerial imagery for disaster response and recovery

Accurate and timely access to data describing disaster impact and extent of damage is key to successful disaster management (a process that includes prevention, mitigation, preparedness, response, and recovery). Airborne data acquisition using helicopter and unmanned aerial vehicle (UAV) helps obtain a bird’s-eye view of disaster-affected areas. However, a major challenge to this approach is robustly processing a large amount of data to identify and map objects of interest on the ground in real-time. The current process is resource-intensive (must be carried out manually) and requires offline computing (through post-processing of aerial videos). This research introduces and evaluates a series of convolutional neural network (CNN) models for ground object detection from aerial views of disaster’s aftermath. These models are capable of recognizing critical ground assets including building roofs (both damaged and undamaged), vehicles, vegetation, debris, and flooded areas. The CNN models are trained on an in-house aerial video dataset (named Volan2018) that is created using web mining techniques. Volan2018 contains eight annotated aerial videos (65,580 frames) collected by drone or helicopter from eight different locations in various hurricanes that struck the United States in 2017–2018. Eight CNN models based on You-Only-Look-Once (YOLO) algorithm are trained by transfer learning, i.e., pre-trained on the COCO/VOC dataset and re-trained on Volan2018 dataset, and achieve 80.69% mAP for high altitude (helicopter footage) and 74.48% for low altitude (drone footage), respectively. This paper also presents a thorough investigation of the effect of camera altitude, data balance, and pre-trained weights on model performance, and finds that models trained and tested on videos taken from similar altitude outperform those trained and tested on videos taken from different altitudes. Moreover, the CNN model pre-trained on the VOC dataset and re-trained on balanced drone video yields the best result in significantly shorter training time.
Shifting trust in construction supply chains through blockchain technology

Purpose Blockchain technology is booming in many industries. Its application in supply chain management is also gradually increasing. Supply chain management (SCM) has long been committed to reducing costs and increasing efficiency and is trying to optimise resources and reduce the sector's fragmentation. Trust has always been an important factor in managing supply chain relationships, and it also affects the efficiency of supply chain operations. To this end, this study aims to examine how trust is affected by the introduction of blockchain technology in construction supply chain management. Design/methodology/approach This study is based on semi-structured interviews and publicly available information from experts in blockchain and construction supply chain management. Through content analysis, the data are analysed thematically to explore how various types of trust, such as system-based, cognition-based and relation-based, are affected by blockchain technology. Findings Blockchain technology provides solutions for data tracking, contracting and transferring resources in supply chain management. These applications help enhance the various sources of trust in SCM and provide supply chain partners with protection mechanisms to avoid the risks and costs of opportunistic behaviour in collaboration, shifting trust from relational to system-based and cognition-based. Research limitations/implications This study focuses only on inter-organisational rather than interpersonal trust and empirical data from experts whose knowledge and cognition could be subjective. Practical implications Leveraging the potential of digitalisation to manage trust requires that leaders and managers actively try to improve contractual arrangements, information sharing and being open to new innovative technologies like blockchain. Social implications From a relational view of supply chain management, the extent to which blockchain technology can develop and spread depends on the readiness of the social capital to accept decentralised governance structures. Originality/value This study builds upon an original data set and discusses features and applications of blockchain technology, explores the sources and dimensions of trust in supply chain management and explains the impact of blockchain technology on trust.
Compressing BERT: Studying the Effects of Weight Pruning on Transfer Learning

Pre-trained feature extractors, such as BERT for natural language processing and VGG for computer vision, have become effective methods for improving deep learning models without requiring more labeled data.While effective, these feature extractors may be prohibitively large for some deployment scenarios.We explore weight pruning for BERT and ask: how does compression during pretraining affect transfer learning?We find that pruning affects transfer learning in three broad regimes.Low levels of pruning (30-40%) do not affect pre-training loss or transfer to downstream tasks at all.Medium levels of pruning increase the pre-training loss and prevent useful pre-training information from being transferred to downstream tasks.High levels of pruning additionally prevent models from fitting downstream datasets, leading to further degradation.Finally, we observe that finetuning BERT on a specific task does not improve its prunability.We conclude that BERT can be pruned once during pre-training rather than separately for each task without affecting performance.