OpenRaiser/Intern-Atlas huggingface dataset:
Validation Obligations: A Novel Approach to Check Compliance between Requirements and their Formal Specification
Venue: arXiv cs.SE 2021
Year: 2021

Traditionally, practitioners use formal methods pre-dominately for one half of the quality-assurance process: verification (do we build the software right?). The other half -- validation (do we build the right software?) -- has been given comparatively little attention. While verification is the core of refinement-based formal methods, where each new refinement step must preserve all properties of its abstract model, validation is usually postponed until the latest stages of the development, when models can be automatically executed. Thus mistakes in requirements or in their interpretation are caught too late: usually at the end of the development process. In this paper, we present a novel approach to check compliance between requirements and their formal refinement-based specification during the earlier stages of development. Our proposed approach -- "validation obligations" -- is based on the simple idea that both verification and validation are an integral part of all refinement steps of a system.
A Fractal Approach to Characterize Emotions in Audio and Visual Domain: A Study on Cross-Modal Interaction
Venue: arXiv cs.SD 2021
Year: 2021

It is already known that both auditory and visual stimulus is able to convey emotions in human mind to different extent. The strength or intensity of the emotional arousal vary depending on the type of stimulus chosen. In this study, we try to investigate the emotional arousal in a cross-modal scenario involving both auditory and visual stimulus while studying their source characteristics. A robust fractal analytic technique called Detrended Fluctuation Analysis (DFA) and its 2D analogue has been used to characterize three (3) standardized audio and video signals quantifying their scaling exponent corresponding to positive and negative valence. It was found that there is significant difference in scaling exponents corresponding to the two different modalities. Detrended Cross Correlation Analysis (DCCA) has also been applied to decipher degree of cross-correlation among the individual audio and visual stimulus. This is the first of its kind study which proposes a novel algorithm with which emotional arousal can be classified in cross-modal scenario using only the source audio and visual signals while also attempting a correlation between them.
Exact Algorithms for Scheduling Problems on Parallel Identical Machines with Conflict Jobs
Venue: arXiv cs.DS 2021
Year: 2021

Machine scheduling problems involving conflict jobs can be seen as a constrained version of the classical scheduling problem, in which some jobs are conflict in the sense that they cannot be proceeded simultaneously on different machines. This conflict constraint naturally arises in several practical applications and has recently received considerable attentions in the research community. In fact, the problem is typically NP-hard (even for approximation) and most of algorithmic results achieved so far have heavily relied on special structures of the underlying graph used to model the conflict-job relation. Our focus is on three objective functions: minimizing the makespan, minimizing the weighted summation of the jobs' completion time, and maximizing the total weights of completed jobs; the first two of which have been intensively studied in the literature. For each objective function considered, we present several mixed integer linear programming models and a constraint programming model, from which we can solve the problems to optimality using dedicated solvers. Binary search-based algorithms are also proposed to solve the makespan problem. The results of numerical experiments performed on randomly generated data sets with up to 32 jobs and 6 machines are reported and analysed to verify the performance of the proposed methods.
Uncertainty Propagation in Convolutional Neural Networks: Technical Report
Venue: arXiv cs.LG 2021
Year: 2021

In this technical report we study the problem of propagation of uncertainty (in terms of variances of given uni-variate normal random variables) through typical building blocks of a Convolutional Neural Network (CNN). These include layers that perform linear operations, such as 2D convolutions, fully-connected, and average pooling layers, as well as layers that act non-linearly on their input, such as the Rectified Linear Unit (ReLU). Finally, we discuss the sigmoid function, for which we give approximations of its first- and second-order moments, as well as the binary cross-entropy loss function, for which we approximate its expected value under normal random inputs.
SelfHAR: Improving Human Activity Recognition through Self-training with Unlabeled Data
Venue: arXiv cs.LG 2021
Year: 2021

Machine learning and deep learning have shown great promise in mobile sensing applications, including Human Activity Recognition. However, the performance of such models in real-world settings largely depends on the availability of large datasets that captures diverse behaviors. Recently, studies in computer vision and natural language processing have shown that leveraging massive amounts of unlabeled data enables performance on par with state-of-the-art supervised models. In this work, we present SelfHAR, a semi-supervised model that effectively learns to leverage unlabeled mobile sensing datasets to complement small labeled datasets. Our approach combines teacher-student self-training, which distills the knowledge of unlabeled and labeled datasets while allowing for data augmentation, and multi-task self-supervision, which learns robust signal-level representations by predicting distorted versions of the input. We evaluated SelfHAR on various HAR datasets and showed state-of-the-art performance over supervised and previous semi-supervised approaches, with up to 12% increase in F1 score using the same number of model parameters at inference. Furthermore, SelfHAR is data-efficient, reaching similar performance using up to 10 times less labeled data compared to supervised approaches. Our work not only achieves state-of-the-art performance in a diverse set of HAR datasets, but also sheds light on how pre-training tasks may affect downstream performance.
End-to-End Delay Guaranteed SFC Deployment: A Multi-level Mapping Approach
Venue: arXiv cs.NI 2021
Year: 2021

Network Function Virtualization (NFV) enables service providers to maximize the business profit via resource-efficient QoS provisioning for customer requested Service Function Chains (SFCs). In recent applications, end-to-end delay is one of the crucial QoS requirement needs to be guaranteed in SFC deployment. While it is considered in the literature, accurate and comprehensive modeling of the delay in the problem of maximizing the service provider's profit has not yet been addressed. In this paper, at first, the problem is formulated as a mixed integer convex programming model. Then, by decomposing the model, a two-level mapping algorithm is developed, that at the first level, maps the functions of the SFCs to virtual network function instances; and in the second level, deploys the instances in the physical network. Simulation results show that the proposed approach achieves a near optimal solution in comparison the performance bound obtained by the optimization model, and also superiors the existing solutions.
An Inquisitive Code Editor for Addressing Novice Programmers' Misconceptions of Program Behavior
Venue: arXiv cs.SE 2021
Year: 2021

Novice programmers face numerous barriers while attempting to learn how to code that may deter them from pursuing a computer science degree or career in software development. In this work, we propose a tool concept to address the particularly challenging barrier of novice programmers holding misconceptions about how their code behaves. Specifically, the concept involves an inquisitive code editor that: (1) identifies misconceptions by periodically prompting the novice programmer with questions about their program's behavior, (2) corrects the misconceptions by generating explanations based on the program's actual behavior, and (3) prevents further misconceptions by inserting test code and utilizing other educational resources. We have implemented portions of the concept as plugins for the Atom code editor and conducted informal surveys with students and instructors. Next steps include deploying the tool prototype to students enrolled in introductory programming courses.
EvoSplit: An evolutionary approach to split a multi-label data set into disjoint subsets
Venue: arXiv cs.LG 2021
Year: 2021

This paper presents a new evolutionary approach, EvoSplit, for the distribution of multi-label data sets into disjoint subsets for supervised machine learning. Currently, data set providers either divide a data set randomly or using iterative stratification, a method that aims to maintain the label (or label pair) distribution of the original data set into the different subsets. Following the same aim, this paper first introduces a single-objective evolutionary approach that tries to obtain a split that maximizes the similarity between those distributions independently. Second, a new multi-objective evolutionary algorithm is presented to maximize the similarity considering simultaneously both distributions (labels and label pairs). Both approaches are validated using well-known multi-label data sets as well as large image data sets currently used in computer vision and machine learning applications. EvoSplit improves the splitting of a data set in comparison to the iterative stratification following different measures: Label Distribution, Label Pair Distribution, Examples Distribution, folds and fold-label pairs with zero positive examples.
Personalized Embedding-based e-Commerce Recommendations at eBay
Venue: arXiv cs.IR 2021
Year: 2021

Recommender systems are an essential component of e-commerce marketplaces, helping consumers navigate massive amounts of inventory and find what they need or love. In this paper, we present an approach for generating personalized item recommendations in an e-commerce marketplace by learning to embed items and users in the same vector space. In order to alleviate the considerable cold-start problem present in large marketplaces, item and user embeddings are computed using content features and multi-modal onsite user activity respectively. Data ablation is incorporated into the offline model training process to improve the robustness of the production system. In offline evaluation using a dataset collected from eBay traffic, our approach was able to improve the Recall@k metric over the Recently-Viewed-Item (RVI) method. This approach to generating personalized recommendations has been launched to serve production traffic, and the corresponding scalable engineering architecture is also presented. Initial A/B test results show that compared to the current personalized recommendation module in production, the proposed method increases the surface rate by $\sim$6\% to generate recommendations for 90\% of listing page impressions.
Defuse: Harnessing Unrestricted Adversarial Examples for Debugging Models Beyond Test Accuracy
Venue: arXiv cs.LG 2021
Year: 2021

We typically compute aggregate statistics on held-out test data to assess the generalization of machine learning models. However, statistics on test data often overstate model generalization, and thus, the performance of deployed machine learning models can be variable and untrustworthy. Motivated by these concerns, we develop methods to automatically discover and correct model errors beyond those available in the data. We propose Defuse, a method that generates novel model misclassifications, categorizes these errors into high-level model bugs, and efficiently labels and fine-tunes on the errors to correct them. To generate misclassified data, we propose an algorithm inspired by adversarial machine learning techniques that uses a generative model to find naturally occurring instances misclassified by a model. Further, we observe that the generative models have regions in their latent space with higher concentrations of misclassifications. We call these regions misclassification regions and find they have several useful properties. Each region contains a specific type of model bug; for instance, a misclassification region for an MNIST classifier contains a style of skinny 6 that the model mistakes as a 1. We can also assign a single label to each region, facilitating low-cost labeling. We propose a method to learn the misclassification regions and use this insight to both categorize errors and correct them. In practice, Defuse finds and corrects novel errors in classifiers. For example, Defuse shows that a high-performance traffic sign classifier mistakes certain 50km/h signs as 80km/h. Defuse corrects the error after fine-tuning while maintaining generalization on the test set.
Robotic Tool Tracking under Partially Visible Kinematic Chain: A Unified Approach
Venue: arXiv cs.RO 2021
Year: 2021

Anytime a robot manipulator is controlled via visual feedback, the transformation between the robot and camera frame must be known. However, in the case where cameras can only capture a portion of the robot manipulator in order to better perceive the environment being interacted with, there is greater sensitivity to errors in calibration of the base-to-camera transform. A secondary source of uncertainty during robotic control are inaccuracies in joint angle measurements which can be caused by biases in positioning and complex transmission effects such as backlash and cable stretch. In this work, we bring together these two sets of unknown parameters into a unified problem formulation when the kinematic chain is partially visible in the camera view. We prove that these parameters are non-identifiable implying that explicit estimation of them is infeasible. To overcome this, we derive a smaller set of parameters we call Lumped Error since it lumps together the errors of calibration and joint angle measurements. A particle filter method is presented and tested in simulation and on two real world robots to estimate the Lumped Error and show the efficiency of this parameter reduction.
Deep Reinforcement Agent for Scheduling in HPC
Venue: arXiv cs.DC 2021
Year: 2021

Cluster scheduler is crucial in high-performance computing (HPC). It determines when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with specific HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a novel, hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling. A unique training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a specific scheduling objective given by system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. The experiments with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 45%.
Towards DeepSentinel: An extensible corpus of labelled Sentinel-1 and -2 imagery and a general-purpose sensor-fusion semantic embedding model
Venue: arXiv cs.CV 2021
Year: 2021

Earth observation offers new insight into anthropogenic changes to nature, and how these changes are effecting (and are effected by) the built environment and the real economy. With the global availability of medium-resolution (10-30m) synthetic aperture radar (SAR) Sentinel-1 and multispectral Sentinel-2 imagery, machine learning can be employed to offer these insights at scale, unbiased to the reporting of companies and countries. In this paper, I introduce DeepSentinel, a data pipeline and experimentation framework for producing general-purpose semantic embeddings of paired Sentinel-1 and Sentinel-2 imagery. I document the development of an extensible corpus of labelled and unlabelled imagery for the purposes of sensor fusion research. With this new dataset I develop a set of experiments applying popular self-supervision methods and encoder architectures to a land cover classification problem. Tile2vec spatial encoding with a self-attention enabled ResNet model outperforms deeper ResNet variants as well as pretraining with variational autoencoding and contrastive loss. All supporting and derived data and code are made publicly available.
Fair Robust Assignment using Redundancy
Venue: arXiv cs.RO 2021
Year: 2021

We study the consideration of fairness in redundant assignment for multi-agent task allocation. It has recently been shown that redundant assignment of agents to tasks provides robustness to uncertainty in task performance. However, the question of how to fairly assign these redundant resources across tasks remains unaddressed. In this paper, we present a novel problem formulation for fair redundant task allocation, which we cast as the optimization of worst-case task costs under a cardinality constraint. Solving this problem optimally is NP-hard. We exploit properties of supermodularity to propose a polynomial-time, near-optimal solution. In supermodular redundant assignment, the use of additional agents always improves task costs. Therefore, we provide a solution set that is $α$ times larger than the cardinality constraint. This constraint relaxation enables our approach to achieve a super-optimal cost by using a sub-optimal assignment size. We derive the sub-optimality bound on this cardinality relaxation, $α$. Additionally, we demonstrate that our algorithm performs near-optimally without the cardinality relaxation. We show simulations of redundant assignments of robots to goal nodes on transport networks with uncertain travel times. Empirically, our algorithm outperforms benchmarks, scales to large problems, and provides improvements in both fairness and average utility.
On Agnostic PAC Learning using $\mathcal{L}_2$-polynomial Regression and Fourier-based Algorithms
Venue: arXiv cs.LG 2021
Year: 2021

We develop a framework using Hilbert spaces as a proxy to analyze PAC learning problems with structural properties. We consider a joint Hilbert space incorporating the relation between the true label and the predictor under a joint distribution $D$. We demonstrate that agnostic PAC learning with 0-1 loss is equivalent to an optimization in the Hilbert space domain. With our model, we revisit the PAC learning problem using methods based on least-squares such as $\mathcal{L}_2$ polynomial regression and Linial's low-degree algorithm. We study learning with respect to several hypothesis classes such as half-spaces and polynomial-approximated classes (i.e., functions approximated by a fixed-degree polynomial). We prove that (under some distributional assumptions) such methods obtain generalization error up to $2opt$ with $opt$ being the optimal error of the class. Hence, we show the tightest bound on generalization error when $opt\leq 0.2$.
K-Hairstyle: A Large-scale Korean Hairstyle Dataset for Virtual Hair Editing and Hairstyle Classification
Venue: arXiv cs.CV 2021
Year: 2021

The hair and beauty industry is a fast-growing industry. This led to the development of various applications, such as virtual hair dyeing or hairstyle transfer, to satisfy the customer's needs. Although several hairstyle datasets are available for these applications, they often consist of a relatively small number of images with low resolution, thus limiting their performance on high-quality hair editing. In response, we introduce a novel large-scale Korean hairstyle dataset, K-hairstyle, containing 500,000 high-resolution images. In addition, K-hairstyle includes various hair attributes annotated by Korean expert hairstylists as well as hair segmentation masks. We validate the effectiveness of our dataset via several applications, such as hair dyeing, hairstyle transfer, and hairstyle classification. K-hairstyle is publicly available at https://psh01087.github.io/K-Hairstyle/.
I Know What You Imported Last Summer: A study of security threats in thePython ecosystem
Venue: arXiv cs.CR 2021
Year: 2021

The popularity of Python has risen rapidly over the past 15 years. It is a major language in some of the most exciting technologies today. This popularity has led to a large ecosystem of third-party packages available via the pip package registry which hosts more than 200,000 packages. These third-party packages can be reused by simply importing the package after installing using package managers like pip. The ease of reuse of third-party software comes with security risks putting millions of users in danger. In this project, we study the ecosystem to analyze this threat. The mature ecosystem of Python has multiple weak spots that we highlight in our project. First, we demonstrate how trivial it is to exploit the Python ecosystem. Then, we systematically analyze dependencies amongst packages, maintainers, and publicly reported security issues. Most attacks are possible only if users install malicious packages. We thus try to analyze and evaluate different methods used by attackers to force incorrect downloads. We quantify your ideas by estimating the potential threat that can be caused by exploiting a popular Python package. We also discuss methods used in the industry to defend against such attacks
Understanding the attitudes, knowledge sharing behaviors and task performance of core developers: A longitudinal study
Venue: arXiv cs.SE 2021
Year: 2021

Context: Prior research has established that a few individuals generally dominate project communication and source code changes during software development, regardless of task assignments at project initiation. Objective: While this phenomenon has been noted, prior research has not sought to understand these dominant individuals. Previous work has found that core communicators are the gatekeepers of their teams' knowledge, and the performance of these members was correlated with their teams' success. Building on this work, we have employed a longitudinal approach to study the way core developers' attitudes, knowledge sharing behaviors and task performance change over the course of their project. Method: We first used Social Network Analysis (SNA) and standard statistical analysis techniques to identify and select artifacts and central practitioners from ten different software development teams. We then applied psycholinguistic analysis and directed content analysis (CA) techniques to interpret the content of these practitioners' messages. Finally, we inspected core developers' activities at various points in time during systems' development. Results: Among our findings, we observe that core developers' attitudes and knowledge sharing behaviors were linked to their involvement in actual software development and the demands of their wider project teams. However, core developers appeared to naturally possess high levels of insightful characteristics. Conclusion: Project performance would likely benefit from strategies aimed at surrounding core developers with other competent communicators. Core developers should also be supported by a wider team who are willing to ask questions and challenge their ideas. Finally, the availability of adequate communication channels would help with maintaining positive team climate especially in distributed developments.(Abridged)