Reviewerly/exHarmony huggingface dataset:
Semi-supervised Crowd Counting via Self-training on Surrogate Tasks

Most existing crowd counting systems rely on the availability of the object location annotation which can be expensive to obtain. To reduce the annotation cost, one attractive solution is to leverage a large number of unlabeled images to build a crowd counting model in semi-supervised fashion. This paper tackles the semi-supervised crowd counting problem from the perspective of feature learning. Our key idea is to leverage the unlabeled images to train a generic feature extractor rather than the entire network of a crowd counter. The rationale of this design is that learning the feature extractor can be more reliable and robust towards the inevitable noisy supervision generated from the unlabeled data. Also, on top of a good feature extractor, it is possible to build a density map regressor with much fewer density map annotations. Specifically, we proposed a novel semi-supervised crowd counting method which is built upon two innovative components: (1) a set of inter-related binary segmentation tasks are derived from the original density map regression task as the surrogate prediction target; (2) the surrogate target predictors are learned from both labeled and unlabeled data by utilizing a proposed self-training scheme which fully exploits the underlying constraints of these binary segmentation tasks. Through experiments, we show that the proposed method is superior over the existing semi-supervised crowd counting method and other representative baselines.
AdaSampling for Positive-Unlabeled and Label Noise Learning With Bioinformatics Applications

Class labels are required for supervised learning but may be corrupted or missing in various applications.In binary classification, for example, when only a subset of positive instances is labeled whereas the remaining are unlabeled, positiveunlabeled learning is required to model from both positive and unlabeled data.Similarly, when class labels are corrupted by mislabeled instances, methods are needed for learning in the presence of class label noise.Here we propose AdaSampling, a framework for both positive-unlabeled learning and learning with class label noise.By iteratively estimating the class mislabeling probability with an adaptive sampling procedure, the proposed method progressively reduces the risk of selecting mislabeled instances for model training and subsequently constructs highly generalisable models even when a large proportion of mislabeled instances is present in the data.We demonstrate the utilities of proposed methods using simulation and benchmark data, and compare them to alternative approaches that are commonly used for positive-unlabeled learning and/or learning with label noise.We then introduce two novel bioinformatics applications where AdaSampling is used to (1) identify kinase-substrates from mass spectrometry-based phosphoproteomics data and (2) predict transcription factor target genes by integrating various nextgeneration sequencing data.
Bottom-up Object Detection by Grouping Extreme and Center Points

With the advent of deep learning, object detection drifted from a bottom-up to a top-down recognition problem. State of the art algorithms enumerate a near-exhaustive list of object locations and classify each into: object or not. In this paper, we show that bottom-up approaches still perform competitively. We detect four extreme points (top-most, left-most, bottom-most, right-most) and one center point of objects using a standard keypoint estimation network. We group the five keypoints into a bounding box if they are geometrically aligned. Object detection is then a purely appearance-based keypoint estimation problem, without region classification or implicit feature learning. The proposed method performs on-par with the state-of-the-art region based detection methods, with a bounding box AP of 43.2% on COCO test-dev. In addition, our estimated extreme points directly span a coarse octagonal mask, with a COCO Mask AP of 18.9%, much better than the Mask AP of vanilla bounding boxes. Extreme point guided segmentation further improves this to 34.6% Mask AP.
EfficientDeRain: Learning Pixel-wise Dilation Filtering for High-Efficiency Single-Image Deraining

Single-image deraining is rather challenging due to the unknown rain model. Existing methods often make specific assumptions of the rain model, which can hardly cover many diverse circumstances in the real world, compelling them to employ complex optimization or progressive refinement. This, however, significantly affects these methods' efficiency and effectiveness for many efficiency-critical applications. To fill this gap, in this paper, we regard the single-image deraining as a general image-enhancing problem and originally propose a model-free deraining method, i.e., EfficientDeRain, which is able to process a rainy image within 10 ms (i.e., around 6 ms on average), over 80 times faster than the state-of-the-art method (i.e., RCDNet), while achieving similar de-rain effects. We first propose novel pixel-wise dilation filtering. In particular, a rainy image is filtered with the pixel-wise kernels estimated from a kernel prediction network, by which suitable multi-scale kernels for each pixel can be efficiently predicted. Then, to eliminate the gap between synthetic and real data, we further propose an effective data augmentation method (i.e., RainMix) that helps to train the network for handling real rainy images. We perform a comprehensive evaluation on both synthetic and real-world rainy datasets to demonstrate the effectiveness and efficiency of our method. We release the model and code in https://github.com/tsingqguo/efficientderain.git.
PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction

Chinese spelling correction (CSC) is a task to detect and correct spelling errors in texts.CSC is essentially a linguistic problem, thus the ability of language understanding is crucial to this task.In this paper, we propose a Pre-trained masked Language mOdel with Misspelled knowledgE (PLOME) for CSC, which jointly learns how to understand language and correct spelling errors.To this end, PLOME masks the chosen tokens with similar characters according to a confusion set rather than the fixed token "[MASK]" as in BERT.Besides character prediction, PLOME also introduces pronunciation prediction to learn the misspelled knowledge on phonic level.Moreover, phonological and visual similarity knowledge is important to this task.PLOME utilizes GRU networks to model such knowledge based on characters' phonics and strokes.Experiments are conducted on widely used benchmarks.Our method achieves superior performance against state-of-the-art approaches by a remarkable margin.We release the source code and pre-trained model for further use by the community 1 .
Global Convergence of Adaptive Gradient Methods for An Over-parameterized Neural Network

Adaptive gradient methods like AdaGrad are widely used in optimizing neural networks. Yet, existing convergence guarantees for adaptive gradient methods require either convexity or smoothness, and, in the smooth setting, only guarantee convergence to a stationary point. We propose an adaptive gradient method and show that for two-layer over-parameterized neural networks -- if the width is sufficiently large (polynomially) -- then the proposed method converges \emph{to the global minimum} in polynomial time, and convergence is robust, \emph{ without the need to fine-tune hyper-parameters such as the step-size schedule and with the level of over-parametrization independent of the training error}. Our analysis indicates in particular that over-parametrization is crucial for the harnessing the full potential of adaptive gradient methods in the setting of neural networks.
Do Transformer Modifications Transfer Across Implementations and Applications?

Sharan Narang, Hyung Won Chung, Yi Tay, Liam Fedus, Thibault Fevry, Michael Matena, Karishma Malkan, Noah Fiedel, Noam Shazeer, Zhenzhong Lan, Yanqi Zhou, Wei Li, Nan Ding, Jake Marcus, Adam Roberts, Colin Raffel. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 2021.
Motivators for Large-Scale Agile Adoption From Management Perspective: A Systematic Literature Review

The use of agile methods for software development has grown to a large extent in the last few years. These methods ensure the quick delivery of software products with minimal cost and user satisfaction. Though these techniques were initially developed for small developmental teams, certain challenges have been observed when these methods are applied on large scale. However, we have conducted a systematic literature review (SLR) for the identification of motivators for adopting agile methods on a large scale from a management perspective. Thus, we have identified a total of 21 motivators for adopting agile methods on a large scale from a management perspective. Among these motivators, some were marked as critical motivators depending on variables, e.g., the factors critical in one variable might not be critical in another variable. The factors which were recorded as critical in all variables are strong executive support, agile development environment training and learning, agile development expertise, team competency, and briefing of top management on agile. Furthermore, we also found that the impact of different motivators was different depending on time and place for project manager guidance, i.e., some motivators were most critical in one region while less critical in another. Similarly, some of the motivators were more critical in previous decades but less critical in recent decades because of different improvements in software processes and technologies. These motivators are also analyzed from different angles, i.e., decade-wise and region wise for project managers guidance. The motivators are extracted from a sample of 58 research papers identified via an SLR process. Finally, we have analyzed the identified motivators based on various variables, such as continents and digital libraries.
Mobile Edge Computing-Based Data-Driven Deep Learning Framework for Anomaly Detection

5G is anticipated to embed an artificial intelligence (AI)-empowerment to adroitly plan, optimize and manage the highly complex network by leveraging data generated at different positions of the network architecture. Outages and situation leading to congestion in a cell pose severe hazard for the network. High false alarms and inadequate accuracy are the major limitations of modern approaches for the anomaly—outage and sudden hype in traffic activity that may result in congestion—detection in mobile cellular networks. This indicates wasting limited resources that ultimately leads to an elevated operational expenditure (OPEX) and also interrupting quality of service (QoS) and quality of experience (QoE). Motivated by the outstanding success of deep learning (DL) technology, our study applies it for detection of the above-mentioned anomalies and also supports mobile edge computing (MEC) paradigm in which core network (CN)’s computations are divided across the cellular infrastructure among different MEC servers (co-located with base stations), to relief the CN. Each server monitors user activities of multiple cells and utilizes <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$L$ </tex-math></inline-formula> -layer feedforward deep neural network (DNN) fueled by real call detail record (CDR) dataset for anomaly detection. Our framework achieved 98.8% accuracy with 0.44% false positive rate (FPR)—notable improvements that surmount the deficiencies of the old studies. The numerical results explicate the usefulness and dominance of our proposed detector.
Short Paper: An Empirical Analysis of Blockchain Forks in Bitcoin

Temporary blockchain forks are part of the regular consensus process in permissionless blockchains such as Bitcoin. As forks can be caused by numerous factors such as latency and miner behavior, their analysis provides insights into these factors, which are otherwise unknown. In this paper we provide an empirical analysis of the announcement and propagation of blocks that led to forks of the Bitcoin blockchain. By analyzing the time differences in the publication of competing blocks, we show that the block propagation delay between miners can be of similar order as the block propagation delay of the average Bitcoin peer. Furthermore, we show that the probability of a block to become part of the main chain increases roughly linearly in the time the block has been published before the competing block. Additionally, we show that the observed frequency of short block intervals between two consecutive blocks mined by the same miner after a fork is conspicuously large. While selfish mining can be a cause for this observation, other causes are also possible. Finally, we show that not only the time difference of the publication of competing blocks but also their propagation speeds vary greatly.
SlotRefine: A Fast Non-Autoregressive Model for Joint Intent Detection and Slot Filling

Slot filling and intent detection are two main tasks in spoken language understanding (SLU) system.In this paper, we propose a novel non-autoregressive model named SlotRefine for joint intent detection and slot filling.Besides, we design a novel two-pass iteration mechanism to handle the uncoordinated slots problem caused by conditional independence of non-autoregressive model.Experiments demonstrate that our model significantly outperforms previous models in slot filling task, while considerably speeding up the decoding (up to ×10.77).In-depth analyses show that 1) pretraining schemes could further enhance our model; 2) two-pass mechanism indeed remedy the uncoordinated slots.
A Knowledge-Aware Sequence-to-Tree Network for Math Word Problem Solving

With the advancements in natural language processing tasks, math word problem solving has received increasing attention.Previous methods have achieved promising results but ignore background common-sense knowledge not directly provided by the problem.In addition, during generation, they focus on local features while neglecting global information.To incorporate external knowledge and global expression information, we propose a novel knowledge-aware sequence-to-tree (KA-S2T) network in which the entities in the problem sequences and their categories are modeled as an entity graph.Based on this entity graph, a graph attention network is used to capture knowledge-aware problem representations.Further, we use a tree-structured decoder with a state aggregation mechanism to capture the long-distance dependency and global expression information.Experimental results on the Math23K dataset revealed that the KA-S2T model can achieve better performance than previously reported best results.
Effective Prediction of Type II Diabetes Mellitus Using Data Mining Classifiers and SMOTE

Diabetes is a metabolic disorder and currently is one of the most appalling diseases that mankind is facing. In diabetic disease, the body does not properly respond to “insulin”, an important hormone that converts sugar into energy needed for the proper functioning of regular life. The disease comes with severe complications on our body as it increases the risk of developing kidney disease, heart disease, eye retinal disease, nerve damage, and blood vessel damage. As per the World Health Organization, about 8.8% of the world was diabetic in the year 2017 and they have projected it to reach nearly 10% by 2045. This study develops a model for diabetic prediction based on data mining classifications techniques. Classification of imbalanced data especially in medical informatics is challenging and was the motivational factor for developing a classifier using a rebalancing algorithm. A two-phase classification model is employed in which the first step is preprocessing the data by use of Synthetic Minority Oversampling Technique (SMOTE), and the second one is feeding five classifiers (Bagging, SVM (Support Vector Machine), MLP (Multi-Layer Perceptron), Simple Logistic and Decision Tree) with the preprocessed data in order to select the best classifier for balanced dataset to predict diabetes. We have achieved an accuracy of 94.7013%, and 0.953 receiver operator characteristics (ROC) curve with decision tree classifier. The validation was achieved via a 10-fold cross validation with an experiment that was conducted on clinical records of 734 patients.
ReachNN*: A Tool for Reachability Analysis of Neural-Network Controlled Systems

We introduce ReachNN*, a tool for reachability analysis of neural-network controlled systems (NNCSs). The theoretical foundation of ReachNN* is the use of Bernstein polynomials to approximate any Lipschitz-continuous neural-network controller with different types of activation functions, with provable approximation error bounds. In addition, the sampling-based error bound estimation in ReachNN* is amenable to GPU-based parallel computing. For further improvement in runtime and error bound estimation, ReachNN* also features optional controller re-synthesis via a technique called verification-aware knowledge distillation (KD) to reduce the Lipschitz constant of the neural-network controller. Experiment results across a set of benchmarks show \(7\times \) to \(422\times \) efficiency improvement over the previous prototype. Moreover, KD enables proof of reachability of NNCSs whose verification results were previously unknown due to large overapproximation errors. An open-source implementation of ReachNN* is available at https://github.com/JmfanBU/ReachNNStar.git.
Experimental underwater quantum key distribution

In recent years, the feasibility of quantum key distribution (QKD) in a water channel has been verified by theory and experiment. Here, we present an experimental investigation of QKD and decoy-state QKD based on the BB84 protocol. The experiment was carried out in a 10 m water tank. The attenuation coefficient of tap water is 0.08/m, which is close to Jerlov Type II seawater. We measured the probability-of-detection matrix of polarization states, and the average fidelity of the four polarization states is up to 98.39%. For the 10 m underwater QKD experiment, 20 MHz optical pulses are generated by modulating the laser diode (LD) and attenuated to an average of 0.1 photons per pulse. The security key rate can reach 563.41 kbits/s and the quantum bit error rate (QBER) is 0.36%. Two decoy states (one of which is the vacuum state) was used in the 10 m underwater decoy-state QKD experiment, and the average QBER of signal state is 0.95%, the security key rate reaches 711.29 kbits/s. According to the parameters of the decoy-state experiment, the maximum secure transmission distance of the underwater decoy-state QKD is predicted to be 19.2 m, while it can be increased to 237.1 m in Jerlov Type I seawater with a lower dark count single photon detector (SPD).
Enabling Efficient 5G NR and 4G LTE Coexistence

Reports on the ability to manage the coexistence of fifth generation mobile communications new radio (5G NR) and 4th generation mobile communications (4G).
Neural Temporal Adaptive Sampling and Denoising

Abstract Despite recent advances in Monte Carlo path tracing at interactive rates, denoised image sequences generated with few samples per‐pixel often yield temporally unstable results and loss of high‐frequency details. We present a novel adaptive rendering method that increases temporal stability and image fidelity of low sample count path tracing by distributing samples via spatio‐temporal joint optimization of sampling and denoising. Adding temporal optimization to the sample predictor enables it to learn spatio‐temporal sampling strategies such as placing more samples in disoccluded regions, tracking specular highlights, etc; adding temporal feedback to the denoiser boosts the effective input sample count and increases temporal stability. The temporal approach also allows us to remove the initial uniform sampling step typically present in adaptive sampling algorithms. The sample predictor and denoiser are deep neural networks that we co‐train end‐to‐end over multiple consecutive frames. Our approach is scalable, allowing trade‐off between quality and performance, and runs at near real‐time rates while achieving significantly better image quality and temporal stability than previous methods.
GangSweep

This work proposes GangSweep, a new backdoor detection framework that leverages the super reconstructive power of Generative Adversarial Networks (GAN) to detect and ''sweep out'' neural backdoors. It is motivated by a series of intriguing empirical investigations, revealing that the perturbation masks generated by GAN are persistent and exhibit interesting statistical properties with low shifting variance and large shifting distance in feature space. Compared with the previous solutions, the proposed approach eliminates the reliance on the access to training data, and shows a high degree of robustness and efficiency for detecting and mitigating a wide range of backdoored models with various settings. Moreover, this is the first work that successfully leverages generative networks to defend against advanced neural backdoors with multiple triggers and their polymorphic forms.
Inaudible Adversarial Perturbations for Targeted Attack in Speaker Recognition

Speaker recognition is a popular topic in biometric authentication and many deep learning approaches have achieved extraordinary performances. However, it has been shown in both image and speech applications that deep neural networks are vulnerable to adversarial examples. In this study, we aim to exploit this weakness to perform targeted adversarial attacks against the x-vector based speaker recognition system. We propose to generate inaudible adversarial perturbations achieving targeted white-box attacks to speaker recognition system based on the psychoacoustic principle of frequency masking. Specifically, we constrict the perturbation under the masking threshold of original audio, instead of using a common l_p norm to measure the perturbations. Experiments on Aishell-1 corpus show that our approach yields up to 98.5% attack success rate to arbitrary gender speaker targets, while retaining indistinguishable attribute to listeners. Furthermore, we also achieve an effective speaker attack when applying the proposed approach to a completely irrelevant waveform, such as music.
Enhancing Automated Essay Scoring Performance via Fine-tuning Pre-trained Language Models with Combination of Regression and Ranking

Automated Essay Scoring (AES) is a critical text regression task that automatically assigns scores to essays based on their writing quality.Recently, the performance of sentence prediction tasks has been largely improved by using Pre-trained Language Models via fusing representations from different layers, constructing an auxiliary sentence, using multitask learning, etc.However, to solve the AES task, previous works utilize shallow neural networks to learn essay representations and constrain calculated scores with regression loss or ranking loss, respectively.Since shallow neural networks trained on limited samples show poor performance to capture deep semantic of texts.And without an accurate scoring function, ranking loss and regression loss measures two different aspects of the calculated scores.To improve AES's performance, we find a new way to fine-tune pre-trained language models with multiple losses of the same task.In this paper, we propose to utilize a pretrained language model to learn text representations first.With scores calculated from the representations, mean square error loss and the batch-wise ListNet loss with dynamic weights constrain the scores simultaneously.We utilize Quadratic Weighted Kappa to evaluate our model on the Automated Student Assessment Prize dataset.Our model outperforms not only state-of-the-art neural models near 3 percent but also the latest statistic model.Especially on the two narrative prompts, our model performs much better than all other state-of-theart models.