Reviewerly/exHarmony huggingface dataset:
Accelerating Serverless Computing by Harvesting Idle Resources

Serverless computing automates fine-grained resource scaling and simplifies the development and deployment of online services with stateless functions. However, it is still non-trivial for users to allocate appropriate resources due to various function types, dependencies, and input sizes. Misconfiguration of resource allocations leaves functions either under-provisioned or over-provisioned and leads to continuous low resource utilization. This paper presents Freyr, a new resource manager (RM) for serverless platforms that maximizes resource efficiency by dynamically harvesting idle resources from over-provisioned functions to under-provisioned functions. Freyr monitors each function’s resource utilization in real-time, detects over-provisioning and under-provisioning, and learns to harvest idle resources safely and accelerates functions efficiently by applying deep reinforcement learning algorithms along with a safeguard mechanism. We have implemented and deployed a Freyr prototype in a 13-node Apache OpenWhisk cluster. Experimental results show that 38.8% of function invocations have idle resources harvested by Freyr, and 39.2% of invocations are accelerated by the harvested resources. Freyr reduces the 99th-percentile function response latency by 32.1% compared to the baseline RMs.
Prototype Feature Extraction for Multi-task Learning

Multi-task learning (MTL) has been widely utilized in various industrial scenarios, such as recommender systems and search engines. MTL can improve learning efficiency and prediction accuracy by exploiting commonalities and differences across tasks. However, MTL is sensitive to relationships among tasks and may have performance degradation in real-world applications, because existing neural-based MTL models often share the same network structures and original input features. To address this issue, we propose a novel multi-task learning model based on Prototype Feature Extraction (PFE) to balance task-specific objectives and inter-task relationships. PFE is a novel component to disentangle features for multiple tasks. To better extract features from original inputs before gating networks, we introduce a new concept, namely prototype feature center, to disentangle features for multiple tasks. The extracted prototype features fuse various features from different tasks to better learn inter-task relationships. PFE updates prototype feature centers and prototype features iteratively. Our model utilizes the learned prototype features and task-specific experts for MTL. We implement PFE on two public datasets. Empirical results show that PFE outperforms state-of-the-art MTL models by extracting prototype features. Furthermore, we deploy PFE in a real-world recommender system (one of the world’s top-tier short video sharing platforms) to showcase that PFE can be widely applied in industrial scenarios.
Unsupervised Representation Learning of Player Behavioral Data with Confidence Guided Masking

Players of online games generate rich behavioral data during gaming. Based on these data, game developers can build a range of data science applications, such as bot detection and social recommendation, to improve the gaming experience. However, the development of such applications requires data cleansing, training sample labeling, feature engineering, and model development, which makes the use of such applications in small and medium-sized game studios still uncommon. While acquiring supervised learning data is costly, unlabeled behavioral logs are often continuously and automatically generated in games. Thus we resort to unsupervised representation learning of player behavioral data to optimize intelligent services in games. Behavioral data has many unique properties, including semantic complexity, excessive length, etc. A worth noting property within raw player behavioral data is that a lot of it is task-irrelevant. For these data characteristics, we introduce a BPE-enhanced compression method and propose a novel adaptive masking strategy called Masking by Token Confidence (MTC) for the Masked Language Modeling (MLM) pre-training task. MTC is designed to increase the masking probabilities of task-relevant tokens. Experiments on four downstream tasks and successful deployment in a world-renowned Massively Multiplayer Online Role-Playing Game (MMORPG) prove the effectiveness of the MTC strategy1.
Threshold Meaningful Secret Image Sharing Scheme Based on QR Code

In the traditional secret image sharing scheme (SIS), the noise-like cover image is vulnerable and thus suspicion is easily aroused by malicious users. Therefore, the meaningful secret image sharing scheme deserves investigation. In this paper, we introduce a new <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <mfenced open="(" close=")" separators="|"> <mrow> <mi>k</mi> <mo>,</mo> <mi>n</mi> </mrow> </mfenced> </math> threshold meaningful secret image sharing scheme based on QR code (MSIS-QR). In the proposed MSIS-QR, the secret image is shared with n QR codes, where each share of the scheme is a valid QR code and can be decoded by a QR code scanner. When k or more shares are collected, the secret image can be losslessly recovered by the Chinese remainder theorem. Experimental results indicate the advantages of our scheme in terms of recovered image quality and decoding time.
Public-Key Cryptosystems and Bounded Distance Decoding of Linear Codes

Error-correcting codes form an important topic in information theory. They are used to correct errors that occur during transmission on a noisy channel. An important method for correcting errors is bounded distance decoding. The public-key cryptosystem is a cryptographic protocol that has two different keys. One of them is a public-key that can be known by everyone, and the other is the private-key only known to the user of the system. The data encrypted with the public-key of a given user can only be decrypted by this user with his or her private-key. In this paper, we propose a public-key cryptosystem based on the error-correcting codes. The decryption is performed by using the bounded distance decoding of the code. For a given code length, dimension, and error-correcting capacity, the new system allows dealing with larger plaintext than other code based public-key cryptosystems.
Unfreeze with Care: Space-Efficient Fine-Tuning of Semantic Parsing Models

Semantic parsing is a key NLP task that maps natural language to structured meaning representations. As in many other NLP tasks, SOTA performance in semantic parsing is now attained by fine-tuning a large pretrained language model (PLM). While effective, this approach is inefficient in the presence of multiple downstream tasks, as a new set of values for all parameters of the PLM needs to be stored for each task separately. Recent work has explored methods for adapting PLMs to downstream tasks while keeping most (or all) of their parameters frozen. We examine two such promising techniques, prefix tuning and bias-term tuning, specifically on semantic parsing. We compare them against each other on two different semantic parsing datasets, and we also compare them against full and partial fine-tuning, both in few-shot and conventional data settings. While prefix tuning is shown to do poorly for semantic parsing tasks off the shelf, we modify it by adding special token embeddings, which results in very strong performance without compromising parameter savings.
Curvature Graph Generative Adversarial Networks

Generative adversarial network (GAN) is widely used for generalized and robust learning on graph data. However, for non-Euclidean graph data, the existing GAN-based graph representation methods generate negative samples by random walk or traverse in discrete space, leading to the information loss of topological properties (e.g. hierarchy and circularity). Moreover, due to the topological heterogeneity (i.e., different densities across the graph structure) of graph data, they suffer from serious topological distortion problems. In this paper, we proposed a novel Curvature Graph Generative Adversarial Networks method, named CurvGAN, which is the first GAN-based graph representation method in the Riemannian geometric manifold. To better preserve the topological properties, we approximate the discrete structure as a continuous Riemannian geometric manifold and generate negative samples efficiently from the wrapped normal distribution. To deal with the topological heterogeneity, we leverage the Ricci curvature for local structures with different topological properties, obtaining to low-distortion representations. Extensive experiments show that CurvGAN consistently and significantly outperforms the state-of-the-art methods across multiple tasks and shows superior robustness and generalization.
Understanding the Practice of Security Patch Management across Multiple Branches in OSS Projects

Since the users of open source software (OSS) projects may not use the latest version all the time, OSS development teams often support code maintenance for old versions through maintaining multiple stable branches. Typically, the developers create a stable branch for each old stable version, deploy security patches on the branch, and release fixed versions at regular intervals. As such, old-version applications in production environments are protected from the disclosed vulnerabilities in a long time. However, the rapidly growing number of OSS vulnerabilities has greatly strained this patch deployment model, and a critical need has arisen for the security community to understand the practice of security patch management across stable branches. In this work, we conduct a large-scale empirical study of stable branches in OSS projects and the security patches deployed on them via investigating 608 stable branches belonging to 26 popular OSS projects as well as more than 2,000 security fixes for 806 CVEs deployed on stable branches.
Preference-Tree-Based Real-Time Recommendation System

In the current era of online information overload, recommendation systems are very useful for helping users locate content that may be of interest to them. A personalized recommendation system presents content based on information such as a user's browsing history and the videos watched. However, information filtering-based recommendation systems are vulnerable to data sparsity and cold-start problems. Additionally, existing recommendation systems suffer from the large overhead incurred in learning regression models used for preference prediction or in selecting groups of similar users. In this study, we propose a preference-tree-based real-time recommendation system that uses various tree models to predict user preferences with a fast runtime. The proposed system predicts preferences based on two balance constants and one similarity threshold to recommend content with a high accuracy while balancing generalized and personalized preferences. The results of comparative experiments and ablation studies confirm that the proposed system can accurately recommend content to users. Specifically, we confirmed that the accuracy and novelty of the recommended content were, respectively, improved by 12.1% and 27.2% compared to existing systems. Furthermore, we verified that the proposed system satisfies real-time requirements and mitigates both cold-start and overfitting problems.
Text Adversarial Attacks and Defenses: Issues, Taxonomy, and Perspectives

Deep neural networks (DNNs) have been widely used in many fields due to their powerful representation learning capabilities. However, they are exposed to serious threats caused by the increasing security issues. Adversarial examples were early discovered in computer vision (CV) field when the models were fooled by perturbing the original inputs, and they also exist in natural language processing (NLP) community. However, unlike the image, the text is discrete and semantic in nature, making the generation of adversarial attacks even more difficult. In this work, we provide a comprehensive overview of adversarial attacks and defenses in the textual domain. First, we introduce the pipeline of NLP, including the vector representations of text, DNN-based victim models, and a formal definition of adversarial attacks, which makes our review self-contained. Second, we propose a novel taxonomy for the existing adversarial attacks and defenses, which is fine-grained and closely aligned with practical applications. Finally, we summarize and discuss the major existing issues and further research directions of text adversarial attacks and defenses.
No Features Needed

Problem: Manual data analysis for extracting useful features in web log anomaly detection can be costly and time-consuming. Automated techniques on the other hand (e.g. Auto-Encoders and CNNs based) usually require supplemental network trainings for feature extractions. Often the systems trained on these features suffer from high False Positive Rates (FPRs) and rectifying them can negatively impact accuracies and add training/tuning delays. Thus manual analysis delays, mandatory supplementary trainings and inferior detection outcomes are the limitations in contemporary web log anomaly detection systems. Proposal: Byte Pair Encoding (BPE) is an automated data representation scheme which requires no training, and only needs a single parsing run for tokenizing available data. Models trained using BPE-based vectors have shown to outperform models trained on similar representations, in tasks such as machine translation (NMT) and language generation (NLG). We therefore propose to use BPE tokens obtained from web log data and consequently vectorized by a pre-trained sequence embedding model for performing web log anomaly detection. Our experiments using two public data sets show that ML models trained on BPE sequence vectors, achieve better results compared to training on both manually and automatically extracted features. Moreover our technique of obtaining log representations is fully automated (requiring only a single hyper-parameter), needs no additional network training and provides representations that give consistent performance across different ML algorithms (a facet absent from feature-based techniques). The only trade-off with our method is an increased upper limit in system memory consumption, as compared to using manual features. This is due to the higher dimensions of the utilized pre-trained embeddings, and reducing it, is our motivation for future work
Adversarial Robustness is Not Enough

The current body of work on adversarial robustness seems to imply that theoretical robustness against adversarial examples leads to more secure systems. In this paper we demonstrate that this premise is erroneous by assessing the strengths and limitations of prominent robustness methods in light of facial authentication, a realistic use case where adversarial perturbations pose a real threat, which allows for a natural reflection on the security gained by the obtained robustness. The main contribution of this paper is an evaluation and critical reflection upon why prominent robustness methods fail to deliver a secure system despite living up to their promises of adding robustness. Our analysis shows that state-of-the-art robustness methods such as Adversarial Training and Guided Complement Entropy struggle to accommodate for two key requirements of facial authentication: (1)~the threat model of facial authentication assumes physical adversarial examples that can be added to the scene as opposed to "classical" adversarial examples that are applied to the digital input. Moreover, an attacker that can directly perturb digital input, does not require adversarial perturbations to impersonate their victim; (2)~robustness properties are only validated for standard classification problems, and often ignore the impact of more practical training paradigms that re-purpose models. Our extensive evaluation of robustness in light of facial authentication allowed us to pinpoint the limitations of these methods. To ensure that the concepts of adversarial robustness and security are more tightly coupled, we recommend to evaluate new defences with applications where adversarial perturbations pose a security threat.
Stochastic-Expert Variational Autoencoder for Collaborative Filtering

Motivated by the recent successes of deep generative models used for collaborative filtering, we propose a novel framework of VAE for collaborative filtering using multiple experts and stochastic expert selection, which allows the model to learn a richer and more complex latent representation of user preferences. In our method, individual experts are sampled stochastically at each user-item interaction which can effectively utilize the variability among multiple experts. While we propose this framework in the context of collaborative filtering, the proposed stochastic expert technique can be used to enhance VAEs in general beyond the application of collaborative filtering. Hence, this novel technique can be of independent interest. We comprehensively evaluate our proposed method, Stochastic-Expert Variational Autoencoder (SE-VAE) on numerical experiments on the real-world benchmark datasets from MovieLens and Netflix and show that it consistently outperforms the existing state-of-the-art methods across all metrics. Our proposed stochastic expert framework is generic and adaptable to any VAE architecture. The experimental results show that the adaptations to various architectures provided performance gains over the existing methods.
Not All Layers Are Equal: A Layer-Wise Adaptive Approach Toward Large-Scale DNN Training

A large-batch training with data parallelism is a widely adopted approach to efficiently train a large deep neural network (DNN) model. Large-batch training, however, often suffers from the problem of the model quality degradation because of its fewer iterations. To alleviate this problem, in general, learning rate (lr) scaling methods have been applied, which increases the learning rate to make an update larger at each iteration. Unfortunately, however, we observe that large-batch training with state-of-the-art lr scaling methods still often degrade the model quality when a batch size crosses a specific limit, rendering such lr methods less useful. To this phenomenon, we hypothesize that existing lr scaling methods overlook the subtle but important differences across “layers” in training, which results in the degradation of the overall model quality. From this hypothesis, we propose a novel approach (LENA) toward the learning rate scaling for large-scale DNN training, employing: (1) a layer-wise adaptive lr scaling to adjust lr for each layer individually, and (2) a layer-wise state-aware warm-up to track the state of the training for each layer and finish its warm-up automatically. The comprehensive evaluation with variations of batch sizes demonstrates that LENA achieves the target accuracy (i.e., the accuracy of single-worker training): (1) within the fewest iterations across different batch sizes (up to 45.2% fewer iterations and 44.7% shorter time than the existing state-of-the-art method), and (2) for training very large-batch sizes, surpassing the limits of all baselines.
Understanding Conflicts in Online Conversations

With the rise of social media, users from across the world are able to connect and converse with each other online. While these connections have facilitated a growth in knowledge, online discussions can also end in acrimonious conflict. Previous computational studies have focused on creating online conflict detection models from inferred labels, primarily examine disagreement but not acrimony, and do not examine the conflict’s emergence. Social science studies have investigated offline conflict, which can differ from its online form, and rarely examines its emergence. The current research aims to understand how online conflicts arise in online personal conversations. Our ground truth is a Facebook tool that allows group members to report conflict to administrators. We contrast discussions ending with a conflict report with paired non-conflict discussions from the same post. We study both user characteristics (e.g., historical user-to-user interactions) and conversation dynamics (e.g., changes in emotional intensity over the course of the conversation). We use logistic regression to identify the features that predict conflict. User characteristics such as the commenter’s gender and previous involvement in negative online activity are strong indicators of conflict. Conversational dynamics, such as an increase in person-oriented discussion, are also important signals of conflict. These results help us understand how conflicts emerge and suggest better detection models and ways to alert group administrators and members early on to mediate the conversation.
Underwater Bearing Only Tracking Using Optimal Observer Maneuver Strategies

This paper considers the problem of tracking a uniform moving source using noisy bearing measurements obtained from a distant observer. Observer trajectory optimization plays a central role in this problem, with the objective to minimize the estimation error of the target state. The Bearing Only Tracking (BOT) of passive targets is mainly focused on the observer maneuver with known trajectories and rarely focused on the future prediction of observer states using adaptive optimization strategies. In this paper, observer paths using one-step ahead optimization based on a performance index are devised which are potentially useful for longer horizon observer trajectory planning in passive tracking. This performance index is the function of source parameters termed as the determinant of Error Covariance Matrix (ECM) which is numerically more efficient than the determinant of Fisher Information Matrix (FIM). The determinant of the FIM requires the calculation of future values for target states and measurements rather than the current values, which is not feasible for Kalman like filters. Therefore, in this paper, the optimization technique is implemented using the state error covariance which is readily available through Kalman filter equations and does require separate numerical calculations. Due to optimal observer maneuver, the performance of the proposed algorithm does not depend on the initial conditions as compared to the conventional tracking methods. The efficiency of the evolutionary algorithm is calculated in terms of range, position and velocity errors and simulation results show 4% fewer estimation errors for ECM based optimization than the determinant of the FIM method.
ALLIE: Active Learning on Large-scale Imbalanced Graphs

Human labeling is time-consuming and costly. This problem is further exacerbated in extremely imbalanced class label scenarios, such as detecting fraudsters in online websites. Active learning selects the most relevant example for human labelers to improve the model performance at a lower cost. However, existing methods for active learning for graph data often assumes that both data and label distributions are balanced. These assumptions fail in extreme rare-class classification scenarios, such as classifying abusive reviews in an e-commerce website.