Reviewerly/exHarmony huggingface dataset:
A Two-Phase Load Balancing Algorithm for Cloud Environment

Load balancing is the phenomenon of distributing workload over various computing resources efficiently. It offers enterprises to efficiently manage different application or workload demands by allocating available resources among different servers, computers, and networks. These services can be accessed and utilized either for home use or for business purposes. Due to the excessive load on the cloud, sometimes it is not feasible to offer all these services to different users efficiently. To solve this excessive load issue, an efficient load balancing technique is used to offer satisfactory services to users as per their expectations also leading to efficient utilization of resources and applications on the cloud platform. This paper presents an enhanced load balancing algorithm named as a two-phase load balancing algorithm. It uses a two-phase checking load balancing approach where the first phase is to divide all virtual machines into two different tables based on their state, that is, available or busy while in the second phase, it equally distributes the loads. The various parameters used to measure the performance of the proposed algorithm are cost, data center processing time, and response time. Cloud analyst simulation tool is used to simulate the algorithm. Simulation results demonstrate superiority of the algorithm with existing ones.
ABNIRML: Analyzing the Behavior of Neural IR Models

Abstract Pretrained contextualized language models such as BERT and T5 have established a new state-of-the-art for ad-hoc search. However, it is not yet well understood why these methods are so effective, what makes some variants more effective than others, and what pitfalls they may have. We present a new comprehensive framework for Analyzing the Behavior of Neural IR ModeLs (ABNIRML), which includes new types of diagnostic probes that allow us to test several characteristics—such as writing styles, factuality, sensitivity to paraphrasing and word order—that are not addressed by previous techniques. To demonstrate the value of the framework, we conduct an extensive empirical study that yields insights into the factors that contribute to the neural model’s gains, and identify potential unintended biases the models exhibit. Some of our results confirm conventional wisdom, for example, that recent neural ranking models rely less on exact term overlap with the query, and instead leverage richer linguistic information, evidenced by their higher sensitivity to word and sentence order. Other results are more surprising, such as that some models (e.g., T5 and ColBERT) are biased towards factually correct (rather than simply relevant) texts. Further, some characteristics vary even for the same base language model, and other characteristics can appear due to random variations during model training.1
ProgressFace: Scale-Aware Progressive Learning for Face Detection

Scale variation stands out as one of key challenges in face detection. Recent attempts have been made to cope with this issue by incorporating image/feature pyramids or adjusting anchor sampling/matching strategies. In this work, we propose a novel scale-aware progressive training mechanism to address large scale variations across faces. Inspired by curriculum learning, our method gradually learns large-to-small face instances. The preceding models learned with easier samples (i.e., large faces) can provide good initialization for succeeding learning with harder samples (i.e., small faces), ultimately deriving a better optimum of face detectors. Moreover, we propose an auxiliary anchor-free enhancement module to facilitate the learning of small faces by supplying positive anchors that may be not covered according to the criterion of IoU overlap. Such anchor-free module will be removed during inference and hence no extra computation cost is introduced. Extensive experimental results demonstrate the superiority of our method compared to the state-of-the-arts on the standard FDDB and WIDER FACE benchmarks. Especially, our ProgressFace-Light with MobileNet-0.25 backbone achieves 87.9% AP on the hard set of WIDER FACE, surpassing largely RetinaFace with the same backbone by 9.7%. Code and our trained face detection models are available at https://github.com/jiashu-zhu/ProgressFace .
FLAMBE: Structural Complexity and Representation Learning of Low Rank MDPs

In order to deal with the curse of dimensionality in reinforcement learning (RL), it is common practice to make parametric assumptions where values or policies are functions of some low dimensional feature space. This work focuses on the representation learning question: how can we learn such features? Under the assumption that the underlying (unknown) dynamics correspond to a low rank transition matrix, we show how the representation learning question is related to a particular non-linear matrix decomposition problem. Structurally, we make precise connections between these low rank MDPs and latent variable models, showing how they significantly generalize prior formulations for representation learning in RL. Algorithmically, we develop FLAMBE, which engages in exploration and representation learning for provably efficient RL in low rank transition models.
Kernel Methods Through the Roof: Handling Billions of Points Efficiently

Kernel methods provide an elegant and principled approach to nonparametric learning, but so far could hardly be used in large scale problems, since naive implementations scale poorly with data size. Recent advances have shown the benefits of a number of algorithmic ideas, for example combining optimization, numerical linear algebra and random projections. Here, we push these efforts further to develop and test a solver that takes full advantage of GPU hardware. Towards this end, we designed a preconditioned gradient solver for kernel methods exploiting both GPU acceleration and parallelization with multiple GPUs, implementing out-of-core variants of common linear algebra operations to guarantee optimal hardware utilization. Further, we optimize the numerical precision of different operations and maximize efficiency of matrix-vector multiplications. As a result we can experimentally show dramatic speedups on datasets with billions of points, while still guaranteeing state of the art performance. Additionally, we make our software available as an easy to use library.
A Joint Multiple Criteria Model in Transfer Learning for Cross-domain Chinese Word Segmentation

Word-level information is important in natural language processing (NLP), especially for the Chinese language due to its high linguistic complexity. Chinese word segmentation (CWS) is an essential task for Chinese downstream NLP tasks. Existing methods have already achieved a competitive performance for CWS on large-scale annotated corpora. However, the accuracy of the method will drop dramatically when it handles an unsegmented text with lots of out-of-vocabulary (OOV) words. In addition, there are many different segmentation criteria for addressing different requirements of downstream NLP tasks. Excessive amounts of models with saving different criteria will generate the explosive growth of the total parameters. To this end, we propose a joint multiple criteria model that shares all parameters to integrate different segmentation criteria into one model. Besides, we utilize a transfer learning method to improve the performance of OOV words. Our proposed method is evaluated by designing comprehensive experiments on multiple benchmark datasets (e.g., Bakeoff 2005, Bakeoff 2008 and SIGHAN 2010). Our method achieves the state-of-the-art performances on all datasets. Importantly, our method also shows a competitive practicability and generalization ability for the CWS task.
Long Document Ranking with Query-Directed Sparse Transformer

The computing cost of transformer selfattention often necessitates breaking long documents to fit in pretrained models in document ranking tasks.In this paper, we design Query-Directed Sparse attention that induces IR-axiomatic structures in transformer self-attention.Our model, QDS-Transformer, enforces the principle properties desired in ranking: local contextualization, hierarchical representation, and query-oriented proximity matching, while it also enjoys efficiency from sparsity.Experiments on one fully supervised and three few-shot TREC document ranking benchmarks demonstrate the consistent and robust advantage of QDS-Transformer over previous approaches, as they either retrofit long documents into BERT or use sparse attention without emphasizing IR principles.We further quantify the computing complexity and demonstrates that our sparse attention with TVM implementation is twice more efficient that the fully-connected selfattention.All source codes, trained model, and predictions of this work are available at https://github.com/hallogameboy/QDS-Transformer.
A generalized neural tangent kernel analysis for two-layer neural networks

A recent breakthrough in deep learning theory shows that the training of over-parameterized deep neural networks can be characterized by a kernel function called \textit{neural tangent kernel} (NTK). However, it is known that this type of results does not perfectly match the practice, as NTK-based analysis requires the network weights to stay very close to their initialization throughout training, and cannot handle regularizers or gradient noises. In this paper, we provide a generalized neural tangent kernel analysis and show that noisy gradient descent with weight decay can still exhibit a kernel-like behavior. This implies that the training loss converges linearly up to a certain accuracy. We also establish a novel generalization error bound for two-layer neural networks trained by noisy gradient descent with weight decay.
Pruning Redundant Mappings in Transformer Models via Spectral-Normalized Identity Prior

Traditional (unstructured) pruning methods for a Transformer model focus on regularizing the individual weights by penalizing them toward zero.In this work, we explore spectralnormalized identity priors (SNIP), a structured pruning approach that penalizes an entire residual module in a Transformer model toward an identity mapping.Our method identifies and discards unimportant non-linear mappings in the residual connections by applying a thresholding operator on the function norm.It is applicable to any structured module, including a single attention head, an entire attention block, or a feed-forward subnetwork.Furthermore, we introduce spectral normalization to stabilize the distribution of the post-activation values of the Transformer layers, further improving the pruning effectiveness of the proposed methodology.We conduct experiments with BERT on 5 GLUE benchmark tasks to demonstrate that SNIP achieves effective pruning results while maintaining comparable performance.Specifically, we improve the performance over the state-of-the-art by 0.5 to 1.0% on average at 50% compression ratio.
Compressive Summarization with Plausibility and Salience Modeling

Compressive summarization systems typically rely on a crafted set of syntactic rules to determine what spans of possible summary sentences can be deleted, then learn a model of what to actually delete by optimizing for content selection (ROUGE).In this work, we propose to relax the rigid syntactic constraints on candidate spans and instead leave compression decisions to two data-driven criteria: plausibility and salience.Deleting a span is plausible if removing it maintains the grammaticality and factuality of a sentence, and spans are salient if they contain important information from the summary.Each of these is judged by a pre-trained Transformer model, and only deletions that are both plausible and not salient can be applied.When integrated into a simple extraction-compression pipeline, our method achieves strong in-domain results on benchmark summarization datasets, and human evaluation shows that the plausibility model generally selects for grammatical and factual deletions.Furthermore, the flexibility of our approach allows it to generalize cross-domain: our system fine-tuned on only 500 samples from a new domain can match or exceed an in-domain extractive model trained on much more data. 1
Apriori algorithm optimization based on Spark platform under big data

To extract useful information from massive data, based on the Spark platform, related techniques of the recommended algorithm were studied. Based on experimental data of a certain scale, the relationship between the various influencing factors of moral education evaluation was discussed and applied to the ranking statistics and correlation analysis functions. The evaluation index system of moral education was obtained. The results showed that Spark performed better than Hadoop in the parallelization implementation of the recommended algorithm. In the case of heterogeneous Spark clusters, the HSATS adaptive task scheduling strategy reduced the completion time of the job, and the utilization of cluster node resources was more reasonable. Therefore, the proposed optimization scheme of the recommendation algorithm improves the evaluation index of the recommendation system.
Open-Edit: Open-Domain Image Manipulation with Open-Vocabulary Instructions

We propose a novel algorithm, named Open-Edit, which is the first attempt on open-domain image manipulation with open-vocabulary instructions. It is a challenging task considering the large variation of image domains and the lack of training supervision. Our approach takes advantage of the unified visual-semantic embedding space pretrained on a general image-caption dataset, and manipulates the embedded visual features by applying text-guided vector arithmetic on the image feature maps. A structure-preserving image decoder then generates the manipulated images from the manipulated feature maps. We further propose an on-the-fly sample-specific optimization approach with cycle-consistency constraints to regularize the manipulated images and force them to preserve details of the source images. Our approach shows promising results in manipulating open-vocabulary color, texture, and high-level attributes for various scenarios of open-domain images (Code is released at https://github.com/xh-liu/Open-Edit ).
Summary of the Cooking Activity Recognition Challenge

Cooking Activity Recognition Challenge [1] is organized as a part of ABC2020 [2]. In this work, we analyze and summarize the approaches of submissions of the Challenge. A dataset consisting of macro and micro activities, collected in a Cooking scenario were opened to the public with a goal of recognizing both of these activities. The participant teams used the dataset and submitted their predictions of test data which was released on March 1st, 2020. The submission of the teams was evaluated rigorously and the winning team achieved about 92.08% averaged accuracy for macro- and micro activities.
The Bitcoin options market: A first look at pricing and risk

This paper offers the first-ever look at Bitcoin options by investigating the wedge between optimum Bitcoin option prices based on classical option valuation models (Black-Scholes-Merton and the Heston-Nandi GARCH (1,1)) and actual premiums at which these options are trading. For this purpose, we use near-the-money call and put options traded on Deribit platform as on 27.01.2020, with the maturities ranging from January 31 to 25 September 2020. In addition, we analyse the risk inherent in Bitcoin options by calculating their Greeks and comparing them to those of traditional commodity options. Pricing results suggest slight overpricing and underpricing for Bitcoin call options with the strike $8,000 maturing on 30.01.2020 and 28.02.2020, respectively. We also find that the Bitcoin options provide much stable deltas over time compared to the other commodity options. This result implies higher insulation from undue price rises with the passage of time for investors in Bitcoin options. Our results are useful to regulators, investors and market managers in better understanding the nuances of the Bitcoin options market in addition to making more informed investment choices.
DeepCIN: Attention-Based Cervical histology Image Classification with Sequential Feature Modeling for Pathologist-Level Accuracy

Background: Cervical cancer is one of the deadliest cancers affecting women globally. Cervical intraepithelial neoplasia (CIN) assessment using histopathological examination of cervical biopsy slides is subject to interobserver variability. Automated processing of digitized histopathology slides has the potential for more accurate classification for CIN grades from normal to increasing grades of pre-malignancy: CIN1, CIN2, and CIN3. Methodology: Cervix disease is generally understood to progress from the bottom (basement membrane) to the top of the epithelium. To model this relationship of disease severity to spatial distribution of abnormalities, we propose a network pipeline, DeepCIN, to analyze high-resolution epithelium images (manually extracted from whole-slide images) hierarchically by focusing on localized vertical regions and fusing this local information for determining Normal/CIN classification. The pipeline contains two classifier networks: (1) a cross-sectional, vertical segment-level sequence generator is trained using weak supervision to generate feature sequences from the vertical segments to preserve the bottom-to-top feature relationships in the epithelium image data and (2) an attention-based fusion network image-level classifier predicting the final CIN grade by merging vertical segment sequences. Results: The model produces the CIN classification results and also determines the vertical segment contributions to CIN grade prediction. Conclusion: Experiments show that DeepCIN achieves pathologist-level CIN classification accuracy.
Online Sequential Extreme Learning Machine Algorithm for Better Predispatch Electricity Price Forecasting Grids

The predispatch price forecast plays a key element in the electricity market. However, such a forecast usually depends on the traditional offline batch-learning technologies, which cannot respond in time to the unexpected changes in the local power system environment. Further, the predispatch local price forecast is often affected by the dynamic price changes from the neighboring regions. This article proposes a novel online learning forecast approach to overcome the above issues to provide a better predispatch price forecast by using the online sequential extreme learning machine (OS-ELM) algorithm. The article proposes a novel data structure in the form of a 2-D orthogonal list and two corresponding OS-ELM modules. One module provides the rolling day-ahead price prediction and prediction intervals using the day-by-day online training update, while the other provides the rolling 30-min prediction using the 2-h-by-2-h online training update. The proposed approach can continuously perceive any unexpected events and any price fluctuations from the neighboring regions in the nonlinear patterns. The proposed approach is validated using simulation studies based on the data from the Australian electricity market, and the simulation results show that the proposed approach can help in improving the forecast accuracy, especially when unexpected changes occur both locally and in the neighboring area.
One DAG to Rule Them All

In this paper, we present novel strategies for optimizing the performance of many binary image processing algorithms. These strategies are collected in an open-source framework, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">graphgen</small> , that is able to automatically generate optimized C++ source code implementing the desired optimizations. Simply starting from a set of rules, the algorithms introduced with the <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">graphgen</small> framework can generate decision trees with minimum average path-length, possibly considering image pattern frequencies, apply state prediction and code compression by the use of directed rooted acyclic graphs (DRAGs). Moreover, the proposed algorithmic solutions allow to combine different optimization techniques and significantly improve performance. Our proposal is showcased on three classical and widely employed algorithms (namely Connected Components Labeling, Thinning, and Contour Tracing). When compared to existing approaches —in 2D and 3D—, implementations using the generated optimal DRAGs perform significantly better than previous state-of-the-art algorithms, both on CPU and GPU.
Unbiased Teacher for Semi-Supervised Object Detection

Semi-supervised learning, i.e., training networks with both labeled and unlabeled data, has made significant progress recently. However, existing works have primarily focused on image classification tasks and neglected object detection which requires more annotation effort. In this work, we revisit the Semi-Supervised Object Detection (SS-OD) and identify the pseudo-labeling bias issue in SS-OD. To address this, we introduce Unbiased Teacher, a simple yet effective approach that jointly trains a student and a gradually progressing teacher in a mutually-beneficial manner. Together with a class-balance loss to downweight overly confident pseudo-labels, Unbiased Teacher consistently improved state-of-the-art methods by significant margins on COCO-standard, COCO-additional, and VOC datasets. Specifically, Unbiased Teacher achieves 6.8 absolute mAP improvements against state-of-the-art method when using 1% of labeled data on MS-COCO, achieves around 10 mAP improvements against the supervised baseline when using only 0.5, 1, 2% of labeled data on MS-COCO.
Large Scale Subject Category Classification of Scholarly Papers With Deep Attentive Neural Networks

Subject categories of scholarly papers generally refer to the knowledge domain(s) to which the papers belong, examples being computer science or physics. Subject category classification is a prerequisite for bibliometric studies, organizing scientific publications for domain knowledge extraction, and facilitating faceted searches for digital library search engines. Unfortunately, many academic papers do not have such information as part of their metadata. Most existing methods for solving this task focus on unsupervised learning that often relies on citation networks. However, a complete list of papers citing the current paper may not be readily available. In particular, new papers that have few or no citations cannot be classified using such methods. Here, we propose a deep attentive neural network (DANN) that classifies scholarly papers using only their abstracts. The network is trained using nine million abstracts from Web of Science (WoS). We also use the WoS schema that covers 104 subject categories. The proposed network consists of two bi-directional recurrent neural networks followed by an attention layer. We compare our model against baselines by varying the architecture and text representation. Our best model achieves micro- <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mrow></mml:math> measure of 0.76 with <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:msub><mml:mi>F</mml:mi><mml:mn>1</mml:mn></mml:msub></mml:mrow></mml:math> of individual subject categories ranging from 0.50 to 0.95. The results showed the importance of retraining word embedding models to maximize the vocabulary overlap and the effectiveness of the attention mechanism. The combination of word vectors with TFIDF outperforms character and sentence level embedding models. We discuss imbalanced samples and overlapping categories and suggest possible strategies for mitigation. We also determine the subject category distribution in CiteSeerX by classifying a random sample of one million academic papers.