OpenRaiser/Intern-Atlas huggingface dataset:
IsiSPL: Toward an Automated Reactive Approach to build Software Product Lines
Venue: arXiv cs.SE 2021
Year: 2021

Over the past decades, Software Product Lines (SPLs) have demonstrated the benefits of systematic reuse to increase software development productivity and software product quality. Of the three adoption approaches, i.e. extractive, proactive and reactive, the reactive approach seems the most suitable for software development in practice. The strength of this approach is that it remains close to classical software development practices. In fact, it avoids a complete analysis of the business domain and its variability (i.e. proactive approach), and avoids requiring a set of product variants that covers this domain (i.e. extractive approach). Nevertheless, despite these advantages, we believe that the main obstacle of the reactive approach adoption is the lack of automation of its re-engineering process. This paper proposes isiSPL: a reactive-based approach that facilitates both construction and maintenance of an SPL. The construction of the SPL is based on a quasi-automatic process. The maintenance of the SPL can be made on a white-box SPL implementation, generated by isiSPL. isiSPL is based on two steps: first, the identification and integration of the artefacts of a newly created product into the structure of the SPL; second, the selection of a set of artefacts and their composition to generate a new product that can either partially or completely meet the requirements provided for a product intended by a developer. We have implemented isiSPL and validated its integration and generation using the two different sets of products from ArgoUML-SPL and Soduko-SPL.
SALYPATH: A Deep-Based Architecture for visual attention prediction
Venue: arXiv cs.CV 2021
Year: 2021

Human vision is naturally more attracted by some regions within their field of view than others. This intrinsic selectivity mechanism, so-called visual attention, is influenced by both high- and low-level factors; such as the global environment (illumination, background texture, etc.), stimulus characteristics (color, intensity, orientation, etc.), and some prior visual information. Visual attention is useful for many computer vision applications such as image compression, recognition, and captioning. In this paper, we propose an end-to-end deep-based method, so-called SALYPATH (SALiencY and scanPATH), that efficiently predicts the scanpath of an image through features of a saliency model. The idea is predict the scanpath by exploiting the capacity of a deep-based model to predict the saliency. The proposed method was evaluated through 2 well-known datasets. The results obtained showed the relevance of the proposed framework comparing to state-of-the-art models.
Using Anomaly Feature Vectors for Detecting, Classifying and Warning of Outlier Adversarial Examples
Venue: arXiv cs.LG 2021
Year: 2021

We present DeClaW, a system for detecting, classifying, and warning of adversarial inputs presented to a classification neural network. In contrast to current state-of-the-art methods that, given an input, detect whether an input is clean or adversarial, we aim to also identify the types of adversarial attack (e.g., PGD, Carlini-Wagner or clean). To achieve this, we extract statistical profiles, which we term as anomaly feature vectors, from a set of latent features. Preliminary findings suggest that AFVs can help distinguish among several types of adversarial attacks (e.g., PGD versus Carlini-Wagner) with close to 93% accuracy on the CIFAR-10 dataset. The results open the door to using AFV-based methods for exploring not only adversarial attack detection but also classification of the attack type and then design of attack-specific mitigation strategies.
Distilling Reinforcement Learning Tricks for Video Games
Venue: arXiv cs.LG 2021
Year: 2021

Reinforcement learning (RL) research focuses on general solutions that can be applied across different domains. This results in methods that RL practitioners can use in almost any domain. However, recent studies often lack the engineering steps ("tricks") which may be needed to effectively use RL, such as reward shaping, curriculum learning, and splitting a large task into smaller chunks. Such tricks are common, if not necessary, to achieve state-of-the-art results and win RL competitions. To ease the engineering efforts, we distill descriptions of tricks from state-of-the-art results and study how well these tricks can improve a standard deep Q-learning agent. The long-term goal of this work is to enable combining proven RL methods with domain-specific tricks by providing a unified software framework and accompanying insights in multiple domains.
Passing a Non-verbal Turing Test: Evaluating Gesture Animations Generated from Speech
Venue: arXiv cs.CV 2021
Year: 2021

People communicate using both speech and non-verbal signals such as gestures, face expression or body pose. Non-verbal signals impact the meaning of the spoken utterance in an abundance of ways. An absence of non-verbal signals impoverishes the process of communication. Yet, when users are represented as avatars, it is difficult to translate non-verbal signals along with the speech into the virtual world without specialized motion-capture hardware. In this paper, we propose a novel, data-driven technique for generating gestures directly from speech. Our approach is based on the application of Generative Adversarial Neural Networks (GANs) to model the correlation rather than causation between speech and gestures. This approach approximates neuroscience findings on how non-verbal communication and speech are correlated. We create a large dataset which consists of speech and corresponding gestures in a 3D human pose format from which our model learns the speaker-specific correlation. We evaluate the proposed technique in a user study that is inspired by the Turing test. For the study, we animate the generated gestures on a virtual character. We find that users are not able to distinguish between the generated and the recorded gestures. Moreover, users are able to identify our synthesized gestures as related or not related to a given utterance. Code and videos are available at https://github.com/mrebol/Gestures-From-Speech
q-Paths: Generalizing the Geometric Annealing Path using Power Means
Venue: arXiv cs.LG 2021
Year: 2021

Many common machine learning methods involve the geometric annealing path, a sequence of intermediate densities between two distributions of interest constructed using the geometric average. While alternatives such as the moment-averaging path have demonstrated performance gains in some settings, their practical applicability remains limited by exponential family endpoint assumptions and a lack of closed form energy function. In this work, we introduce $q$-paths, a family of paths which is derived from a generalized notion of the mean, includes the geometric and arithmetic mixtures as special cases, and admits a simple closed form involving the deformed logarithm function from nonextensive thermodynamics. Following previous analysis of the geometric path, we interpret our $q$-paths as corresponding to a $q$-exponential family of distributions, and provide a variational representation of intermediate densities as minimizing a mixture of $α$-divergences to the endpoints. We show that small deviations away from the geometric path yield empirical gains for Bayesian inference using Sequential Monte Carlo and generative model evaluation using Annealed Importance Sampling.
Case study of Innovative Teaching Practices and their Impact for Electrical Engineering Courses during COVID-19 Pandemic
Venue: arXiv cs.CY 2021
Year: 2021

Due to the COVID-19 pandemic, there was an urgent need to move to online teaching and develop innovations to guarantee the Student Learning Outcomes (SLOs) are being fulfilled. The contributions of this paper are two-fold: the effects of an experimented teaching strategy, i.e. multi-course project-based learning (MPL) approach, are presented followed with online assessment techniques investigation for senior level electrical engineering (EE) courses at Qatar University. The course project of the senior course was designed in such a way that it helps in simultaneously attaining the objectives of the senior and capstone courses, that the students were taking at the same time. It is known that the MPL approach enhances the critical thinking capacity of students which is also a major outcome of Education for Sustainable Development (ESD). The developed project ensures the fulfillment of a series of SLOs, that are concentrated on soft engineering and project management skills. The difficulties of adopting the MPL method for the senior level courses are in aligning the project towards fulfilling the learning outcomes of every individual course. The study also provides the students feedback on online assessment techniques incorporated with the MPL, due to online teaching during COVID-19 pandemic. In order to provide a benchmark and to highlight the obtained results, the innovative teaching approaches were compared to conventional methods taught on the same senior course in a previous semester. Based on the feedback from teachers and students from previously conducted case study it was believed that the MPL approach would support the students. With the statistical analysis (Chi-square, two-tailed T statistics and hypothesis testing using z-test) it can be concluded that the MPL and online assessment actually help to achieve better attainment of the SLOs, even during a pandemic situation.
The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models
Venue: arXiv cs.LG 2021
Year: 2021

Supervised learning models often make systematic errors on rare subsets of the data. When these subsets correspond to explicit labels in the data (e.g., gender, race) such poor performance can be identified straightforwardly. This paper introduces a method for discovering systematic errors that do not correspond to such explicitly labelled subgroups. The key idea is that similar inputs tend to have similar representations in the final hidden layer of a neural network. We leverage this structure by "shining a spotlight" on this representation space to find contiguous regions where the model performs poorly. We show that the spotlight surfaces semantically meaningful areas of weakness in a wide variety of existing models spanning computer vision, NLP, and recommender systems.
1st Place Solutions for UG2+ Challenge 2021 -- (Semi-)supervised Face detection in the low light condition
Venue: arXiv cs.CV 2021
Year: 2021

In this technical report, we briefly introduce the solution of our team "TAL-ai" for (Semi-) supervised Face detection in the low light condition in UG2+ Challenge in CVPR 2021. By conducting several experiments with popular image enhancement methods and image transfer methods, we pulled the low light image and the normal image to a more closer domain. And it is observed that using these data to training can achieve better performance. We also adapt several popular object detection frameworks, e.g., DetectoRS, Cascade-RCNN, and large backbone like Swin-transformer. Finally, we ensemble several models which achieved mAP 74.89 on the testing set, ranking 1st on the final leaderboard.
Decision tree heuristics can fail, even in the smoothed setting
Venue: arXiv cs.LG 2021
Year: 2021

Greedy decision tree learning heuristics are mainstays of machine learning practice, but theoretical justification for their empirical success remains elusive. In fact, it has long been known that there are simple target functions for which they fail badly (Kearns and Mansour, STOC 1996). Recent work of Brutzkus, Daniely, and Malach (COLT 2020) considered the smoothed analysis model as a possible avenue towards resolving this disconnect. Within the smoothed setting and for targets $f$ that are $k$-juntas, they showed that these heuristics successfully learn $f$ with depth-$k$ decision tree hypotheses. They conjectured that the same guarantee holds more generally for targets that are depth-$k$ decision trees. We provide a counterexample to this conjecture: we construct targets that are depth-$k$ decision trees and show that even in the smoothed setting, these heuristics build trees of depth $2^{Ω(k)}$ before achieving high accuracy. We also show that the guarantees of Brutzkus et al. cannot extend to the agnostic setting: there are targets that are very close to $k$-juntas, for which these heuristics build trees of depth $2^{Ω(k)}$ before achieving high accuracy.
An Experience Report on Machine Learning Reproducibility: Guidance for Practitioners and TensorFlow Model Garden Contributors
Venue: arXiv cs.SE 2021
Year: 2021

Machine learning techniques are becoming a fundamental tool for scientific and engineering progress. These techniques are applied in contexts as diverse as astronomy and spam filtering. However, correctly applying these techniques requires careful engineering. Much attention has been paid to the technical potential; relatively little attention has been paid to the software engineering process required to bring research-based machine learning techniques into practical utility. Technology companies have supported the engineering community through machine learning frameworks such as TensorFLow and PyTorch, but the details of how to engineer complex machine learning models in these frameworks have remained hidden. To promote best practices within the engineering community, academic institutions and Google have partnered to launch a Special Interest Group on Machine Learning Models (SIGMODELS) whose goal is to develop exemplary implementations of prominent machine learning models in community locations such as the TensorFlow Model Garden (TFMG). The purpose of this report is to define a process for reproducing a state-of-the-art machine learning model at a level of quality suitable for inclusion in the TFMG. We define the engineering process and elaborate on each step, from paper analysis to model release. We report on our experiences implementing the YOLO model family with a team of 26 student researchers, share the tools we developed, and describe the lessons we learned along the way.
Deep learning-based statistical noise reduction for multidimensional spectral data
Venue: arXiv cs.LG 2021
Year: 2021

In spectroscopic experiments, data acquisition in multi-dimensional phase space may require long acquisition time, owing to the large phase space volume to be covered. In such case, the limited time available for data acquisition can be a serious constraint for experiments in which multidimensional spectral data are acquired. Here, taking angle-resolved photoemission spectroscopy (ARPES) as an example, we demonstrate a denoising method that utilizes deep learning as an intelligent way to overcome the constraint. With readily available ARPES data and random generation of training data set, we successfully trained the denoising neural network without overfitting. The denoising neural network can remove the noise in the data while preserving its intrinsic information. We show that the denoising neural network allows us to perform similar level of second-derivative and line shape analysis on data taken with two orders of magnitude less acquisition time. The importance of our method lies in its applicability to any multidimensional spectral data that are susceptible to statistical noise.
Construction and Adaptability Analysis of User's Preference Models Based on Check-in Data in LBSN
Venue: arXiv cs.HC 2021
Year: 2021

With the widespread use of mobile phones, users can share their location anytime, anywhere, as a form of check-in data. These data reflect user preferences. Furthermore, the preference rules for different users vary. How to discover a user's preference from their related information and how to validate whether a preference model is suited to a user is important for providing a suitable service to the user. This study provides four main contributions. First, multiple preference models from different views for each user are constructed. Second, an algorithm is proposed to validate whether a preference model is applicable to the user by calculating the stability value of the user's long-term check-in data for each model. Third, a unified model, i.e., a multi-channel convolutional neural network is used to characterize this applicability. Finally, three datasets from multiple sources are used to verify the validity of the method, the results of which show the effectiveness of the method.
HO-3D_v3: Improving the Accuracy of Hand-Object Annotations of the HO-3D Dataset
Venue: arXiv cs.CV 2021
Year: 2021

HO-3D is a dataset providing image sequences of various hand-object interaction scenarios annotated with the 3D pose of the hand and the object and was originally introduced as HO-3D_v2. The annotations were obtained automatically using an optimization method, 'HOnnotate', introduced in the original paper. HO-3D_v3 provides more accurate annotations for both the hand and object poses thus resulting in better estimates of contact regions between the hand and the object. In this report, we elaborate on the improvements to the HOnnotate method and provide evaluations to compare the accuracy of HO-3D_v2 and HO-3D_v3. HO-3D_v3 results in 4mm higher accuracy compared to HO-3D_v2 for hand poses while exhibiting higher contact regions with the object surface.
Online Multi-Agent Forecasting with Interpretable Collaborative Graph Neural Network
Venue: arXiv cs.AI 2021
Year: 2021

This paper considers predicting future statuses of multiple agents in an online fashion by exploiting dynamic interactions in the system. We propose a novel collaborative prediction unit (CoPU), which aggregates the predictions from multiple collaborative predictors according to a collaborative graph. Each collaborative predictor is trained to predict the status of an agent by considering the impact of another agent. The edge weights of the collaborative graph reflect the importance of each predictor. The collaborative graph is adjusted online by multiplicative update, which can be motivated by minimizing an explicit objective. With this objective, we also conduct regret analysis to indicate that, along with training, our CoPU achieves similar performance with the best individual collaborative predictor in hindsight. This theoretical interpretability distinguishes our method from many other graph networks. To progressively refine predictions, multiple CoPUs are stacked to form a collaborative graph neural network. Extensive experiments are conducted on three tasks: online simulated trajectory prediction, online human motion prediction and online traffic speed prediction, and our methods outperform state-of-the-art works on the three tasks by 28.6%, 17.4% and 21.0% on average, respectively.
Theory of Deep Convolutional Neural Networks III: Approximating Radial Functions
Venue: arXiv cs.LG 2021
Year: 2021

We consider a family of deep neural networks consisting of two groups of convolutional layers, a downsampling operator, and a fully connected layer. The network structure depends on two structural parameters which determine the numbers of convolutional layers and the width of the fully connected layer. We establish an approximation theory with explicit approximation rates when the approximated function takes a composite form $f\circ Q$ with a feature polynomial $Q$ and a univariate function $f$. In particular, we prove that such a network can outperform fully connected shallow networks in approximating radial functions with $Q(x) =|x|^2$, when the dimension $d$ of data from $\mathbb{R}^d$ is large. This gives the first rigorous proof for the superiority of deep convolutional neural networks in approximating functions with special structures. Then we carry out generalization analysis for empirical risk minimization with such a deep network in a regression framework with the regression function of the form $f\circ Q$. Our network structure which does not use any composite information or the functions $Q$ and $f$ can automatically extract features and make use of the composite nature of the regression function via tuning the structural parameters. Our analysis provides an error bound which decreases with the network depth to a minimum and then increases, verifying theoretically a trade-off phenomenon observed for network depths in many practical applications.
Data Centric Domain Adaptation for Historical Text with OCR Errors
Venue: arXiv cs.CL 2021
Year: 2021

We propose new methods for in-domain and cross-domain Named Entity Recognition (NER) on historical data for Dutch and French. For the cross-domain case, we address domain shift by integrating unsupervised in-domain data via contextualized string embeddings; and OCR errors by injecting synthetic OCR errors into the source domain and address data centric domain adaptation. We propose a general approach to imitate OCR errors in arbitrary input data. Our cross-domain as well as our in-domain results outperform several strong baselines and establish state-of-the-art results. We publish preprocessed versions of the French and Dutch Europeana NER corpora.
A Novel Deep Reinforcement Learning Based Stock Direction Prediction using Knowledge Graph and Community Aware Sentiments
Venue: arXiv cs.AI 2021
Year: 2021

Stock market prediction has been an important topic for investors, researchers, and analysts. Because it is affected by too many factors, stock market prediction is a difficult task to handle. In this study, we propose a novel method that is based on deep reinforcement learning methodologies for the direction prediction of stocks using sentiments of community and knowledge graph. For this purpose, we firstly construct a social knowledge graph of users by analyzing relations between connections. After that, time series analysis of related stock and sentiment analysis is blended with deep reinforcement methodology. Turkish version of Bidirectional Encoder Representations from Transformers (BerTurk) is employed to analyze the sentiments of the users while deep Q-learning methodology is used for the deep reinforcement learning side of the proposed model to construct the deep Q network. In order to demonstrate the effectiveness of the proposed model, Garanti Bank (GARAN), Akbank (AKBNK), Türkiye İş Bankası (ISCTR) stocks in Istanbul Stock Exchange are used as a case study. Experiment results show that the proposed novel model achieves remarkable results for stock market prediction task.
Hybrid Supervision Learning for Pathology Whole Slide Image Classification
Venue: arXiv cs.CV 2021
Year: 2021

Weak supervision learning on classification labels has demonstrated high performance in various tasks, while a few pixel-level fine annotations are also affordable. Naturally a question comes to us that whether the combination of pixel-level (e.g., segmentation) and image level (e.g., classification) annotation can introduce further improvement. However in computational pathology this is a difficult task for this reason: High resolution of whole slide images makes it difficult to do end-to-end classification model training, which is challenging to research of weak or hybrid supervision learning in the past. To handle this problem, we propose a hybrid supervision learning framework for this kind of high resolution images with sufficient image-level coarse annotations and a few pixel-level fine labels. This framework, when applied in training patch model, can carefully make use of coarse image-level labels to refine generated pixel-level pseudo labels. Complete strategy is proposed to suppress pixel-level false positives and false negatives. A large hybrid annotated dataset is used to evaluate the effectiveness of hybrid supervision learning. By extracting pixel-level pseudo labels in initially image-level labeled samples, we achieve 5.2% higher specificity than purely training on existing labels while retaining 100% sensitivity, in the task of image-level classification to be positive or negative.