OpenRaiser/Intern-Atlas huggingface dataset:
Analysis of memory consumption by neural networks based on hyperparameters
Venue: arXiv cs.LG 2021
Year: 2021

Deep learning models are trained and deployed in multiple domains. Increasing usage of deep learning models alarms the usage of memory consumed while computation by deep learning models. Existing approaches for reducing memory consumption like model compression, hardware changes are specific. We propose a generic analysis of memory consumption while training deep learning models in comparison with hyperparameters used for training. Hyperparameters which includes the learning rate, batchsize, number of hidden layers and depth of layers decide the model performance, accuracy of the model. We assume the optimizers and type of hidden layers as a known values. The change in hyperparamaters and the number of hidden layers are the variables considered in this proposed approach. For better understanding of the computation cost, this proposed analysis studies the change in memory consumption with respect to hyperparameters as main focus. This results in general analysis of memory consumption changes during training when set of hyperparameters are altered.
A Machine Learning Framework Towards Transparency in Experts' Decision Quality
Venue: arXiv cs.LG 2021
Year: 2021

Expert workers make non-trivial decisions with significant implications. Experts' decision accuracy is thus a fundamental aspect of their judgment quality, key to both management and consumers of experts' services. Yet, in many important settings, transparency in experts' decision quality is rarely possible because ground truth data for evaluating the experts' decisions is costly and available only for a limited set of decisions. Furthermore, different experts typically handle exclusive sets of decisions, and thus prior solutions that rely on the aggregation of multiple experts' decisions for the same instance are inapplicable. We first formulate the problem of estimating experts' decision accuracy in this setting and then develop a machine-learning-based framework to address it. Our method effectively leverages both abundant historical data on workers' past decisions, and scarce decision instances with ground truth information. We conduct extensive empirical evaluations of our method's performance relative to alternatives using both semi-synthetic data based on publicly available datasets, and purposefully compiled dataset on real workers' decisions. The results show that our approach is superior to existing alternatives across diverse settings, including different data domains, experts' qualities, and the amount of ground truth data. To our knowledge, this paper is the first to posit and address the problem of estimating experts' decision accuracies from historical data with scarcely available ground truth, and it is the first to offer comprehensive results for this problem setting, establishing the performances that can be achieved across settings, as well as the state-of-the-art performance on which future work can build.
Native versus Overlay-based NDN over Wi-Fi 6 for the Internet of Vehicles
Venue: arXiv cs.NI 2021
Year: 2021

Internet of Vehicles (IoV) is a cornerstone building block of smart cities to provide better traffic safety and mobile infotainment. Recently, improved efficiency in WLAN-based dense scenarios has become widespread through Wi-Fi 6, a license-free spectrum technology that can complement the cellular-based infrastructure for IoV. In addition, Named Data Networking (NDN) is a promising Internet architecture to accomplish content distribution in dynamic IoV scenarios. However, NDN deployments, i.e., native (clean-slate) and overlay (running on top of IP stack), require further investigation of their performance over wireless networks, particularly regarding the IoV scenario. This paper performs a comparative simulation-based study of these NDN deployments over Wi-Fi 6 for IoV using real vehicular traces. To the best of our knowledge, this is the first effort that extends ndnSIM 2 with an overlay-based NDN implementation and that compares it with the native approach. Results show that the overlay-based NDN consistently outperforms the native one, reaching around 99% of requests satisfied, against only 42.35% in the best case of native deployment.
Categorizing Items with Short and Noisy Descriptions using Ensembled Transferred Embeddings
Venue: arXiv cs.LG 2021
Year: 2021

Item categorization is a machine learning task which aims at classifying e-commerce items, typically represented by textual attributes, to their most suitable category from a predefined set of categories. An accurate item categorization system is essential for improving both the user experience and the operational processes of the company. In this work, we focus on item categorization settings in which the textual attributes representing items are noisy and short, and labels (i.e., accurate classification of items into categories) are not available. In order to cope with such settings, we propose a novel learning framework, Ensembled Transferred Embeddings (ETE), which relies on two key ideas: 1) labeling a relatively small sample of the target dataset, in a semi-automatic process, and 2) leveraging other datasets from related domains or related tasks that are large-scale and labeled, to extract "transferable embeddings". Evaluation of ETE on a large-scale real-world dataset provided to us by PayPal, shows that it significantly outperforms traditional as well as state-of-the-art item categorization methods.
To Recommend or Not? A Model-Based Comparison of Item-Matching Processes
Venue: arXiv cs.CY 2021
Year: 2021

Recommender systems are central to modern online platforms, but a popular concern is that they may be pulling society in dangerous directions (e.g., towards filter bubbles). However, a challenge with measuring the effects of recommender systems is how to compare user outcomes under these systems to outcomes under a credible counterfactual world without such systems. We take a model-based approach to this challenge, introducing a dichotomy of process models that we can compare: (1) a "recommender" model describing a generic item-matching process under a personalized recommender system and (2) an "organic" model describing a baseline counterfactual where users search for items without the mediation of any system. Our key finding is that the recommender and organic models result in dramatically different outcomes at both the individual and societal level, as supported by theorems and simulation experiments with real data. The two process models also induce different trade-offs during inference, where standard performance-improving techniques such as regularization/shrinkage have divergent effects. Shrinkage improves the mean squared error of matches in both settings, as expected, but at the cost of less diverse (less radical) items chosen in the recommender model but more diverse (more radical) items chosen in the organic model. These findings provide a formal language for how recommender systems may be fundamentally altering how we search for and interact with content, in a world increasingly mediated by such systems.
Adaptive Bridge between Training and Inference for Dialogue
Venue: arXiv cs.CL 2021
Year: 2021

Although exposure bias has been widely studied in some NLP tasks, it faces its unique challenges in dialogue response generation, the representative one-to-various generation scenario. In real human dialogue, there are many appropriate responses for the same context, not only with different expressions, but also with different topics. Therefore, due to the much bigger gap between various ground-truth responses and the generated synthetic response, exposure bias is more challenging in dialogue generation task. What's more, as MLE encourages the model to only learn the common words among different ground-truth responses, but ignores the interesting and specific parts, exposure bias may further lead to the common response generation problem, such as "I don't know" and "HaHa?" In this paper, we propose a novel adaptive switching mechanism, which learns to automatically transit between ground-truth learning and generated learning regarding the word-level matching score, such as the cosine similarity. Experimental results on both Chinese STC dataset and English Reddit dataset, show that our adaptive method achieves a significant improvement in terms of metric-based evaluation and human evaluation, as compared with the state-of-the-art exposure bias approaches. Further analysis on NMT task also shows that our model can achieve a significant improvement.
How to Schedule Near-Optimally under Real-World Constraints
Venue: arXiv cs.PF 2021
Year: 2021

Scheduling is a critical part of practical computer systems, and scheduling has also been extensively studied from a theoretical perspective. Unfortunately, there is a gap between theory and practice, as the optimal scheduling policies presented by theory can be difficult or impossible to perfectly implement in practice. In this work, we use recent breakthroughs in queueing theory to begin to bridge this gap. We show how to translate theoretically optimal policies -- which provably minimize mean response time (a.k.a. latency) -- into near-optimal policies that are easily implemented in practical settings. Specifically, we handle the following real-world constraints: - We show how to schedule in systems where job sizes (a.k.a. running time) are unknown, or only partially known. We do so using simple policies that achieve performance very close to the much more complicated theoretically optimal policies. - We show how to schedule in systems that have only a limited number of priority levels available. We show how to adapt theoretically optimal policies to this constrained setting and determine how many levels we need for near-optimal performance. - We show how to schedule in systems where job preemption can only happen at specific checkpoints. Adding checkpoints allows for smarter scheduling, but each checkpoint incurs time overhead. We give a rule of thumb that near-optimally balances this tradeoff.
Rethinking Generalization Performance of Surgical Phase Recognition with Expert-Generated Annotations
Venue: arXiv cs.CV 2021
Year: 2021

As the area of application of deep neural networks expands to areas requiring expertise, e.g., in medicine and law, more exquisite annotation processes for expert knowledge training are required. In particular, it is difficult to guarantee generalization performance in the clinical field in the case of expert knowledge training where opinions may differ even among experts on annotations. To raise the issue of the annotation generation process for expertise training of CNNs, we verified the annotations for surgical phase recognition of laparoscopic cholecystectomy and subtotal gastrectomy for gastric cancer. We produce calibrated annotations for the seven phases of cholecystectomy by analyzing the discrepancies of previously annotated labels and by discussing the criteria of surgical phases. For gastrectomy for gastric cancer has more complex twenty-one surgical phases, we generate consensus annotation by the revision process with five specialists. By training the CNN-based surgical phase recognition networks with revised annotations, we achieved improved generalization performance over models trained with original annotation under the same cross-validation settings. We showed that the expertise data annotation pipeline for deep neural networks should be more rigorous based on the type of problem to apply clinical field.
Improving Face Recognition with Large Age Gaps by Learning to Distinguish Children
Venue: arXiv cs.CV 2021
Year: 2021

Despite the unprecedented improvement of face recognition, existing face recognition models still show considerably low performances in determining whether a pair of child and adult images belong to the same identity. Previous approaches mainly focused on increasing the similarity between child and adult images of a given identity to overcome the discrepancy of facial appearances due to aging. However, we observe that reducing the similarity between child images of different identities is crucial for learning distinct features among children and thus improving face recognition performance in child-adult pairs. Based on this intuition, we propose a novel loss function called the Inter-Prototype loss which minimizes the similarity between child images. Unlike the previous studies, the Inter-Prototype loss does not require additional child images or training additional learnable parameters. Our extensive experiments and in-depth analyses show that our approach outperforms existing baselines in face recognition with child-adult pairs. Our code and newly-constructed test sets of child-adult pairs are available at https://github.com/leebebeto/Inter-Prototype.
Explainable Landscape-Aware Optimization Performance Prediction
Venue: arXiv cs.NE 2021
Year: 2021

Efficient solving of an unseen optimization problem is related to appropriate selection of an optimization algorithm and its hyper-parameters. For this purpose, automated algorithm performance prediction should be performed that in most commonly-applied practices involves training a supervised ML algorithm using a set of problem landscape features. However, the main issue of training such models is their limited explainability since they only provide information about the joint impact of the set of landscape features to the end prediction results. In this study, we are investigating explainable landscape-aware regression models where the contribution of each landscape feature to the prediction of the optimization algorithm performance is estimated on a global and local level. The global level provides information about the impact of the feature across all benchmark problems' instances, while the local level provides information about the impact on a specific problem instance. The experimental results are obtained using the COCO benchmark problems and three differently configured modular CMA-ESs. The results show a proof of concept that different set of features are important for different problem instances, which indicates that further personalization of the landscape space is required when training an automated algorithm performance prediction model.
EXSCALATE: An extreme-scale in-silico virtual screening platform to evaluate 1 trillion compounds in 60 hours on 81 PFLOPS supercomputers
Venue: arXiv cs.DC 2021
Year: 2021

The social and economic impact of the COVID-19 pandemic demands the reduction of the time required to find a therapeutic cure. In the contest of urgent computing, we re-designed the Exscalate molecular docking platform to benefit from heterogeneous computation nodes and to avoid scaling issues. We deployed the Exscalate platform on two top European supercomputers (CINECA-Marconi100 and ENI-HPC5), with a combined computational power of 81 PFLOPS, to evaluate the interaction between 70 billions of small molecules and 15 binding-sites of 12 viral proteins of Sars-Cov2. The experiment lasted 60 hours and overall it performed a trillion of evaluations.
WebFed: Cross-platform Federated Learning Framework Based on Web Browser with Local Differential Privacy
Venue: arXiv cs.CR 2021
Year: 2021

For data isolated islands and privacy issues, federated learning has been extensively invoking much interest since it allows clients to collaborate on training a global model using their local data without sharing any with a third party. However, the existing federated learning frameworks always need sophisticated condition configurations (e.g., sophisticated driver configuration of standalone graphics card like NVIDIA, compile environment) that bring much inconvenience for large-scale development and deployment. To facilitate the deployment of federated learning and the implementation of related applications, we innovatively propose WebFed, a novel browser-based federated learning framework that takes advantage of the browser's features (e.g., Cross-platform, JavaScript Programming Features) and enhances the privacy protection via local differential privacy mechanism. Finally, We conduct experiments on heterogeneous devices to evaluate the performance of the proposed WebFed framework.
Reimagine BiSeNet for Real-Time Domain Adaptation in Semantic Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Semantic segmentation models have reached remarkable performance across various tasks. However, this performance is achieved with extremely large models, using powerful computational resources and without considering training and inference time. Real-world applications, on the other hand, necessitate models with minimal memory demands, efficient inference speed, and executable with low-resources embedded devices, such as self-driving vehicles. In this paper, we look at the challenge of real-time semantic segmentation across domains, and we train a model to act appropriately on real-world data even though it was trained on a synthetic realm. We employ a new lightweight and shallow discriminator that was specifically created for this purpose. To the best of our knowledge, we are the first to present a real-time adversarial approach for assessing the domain adaption problem in semantic segmentation. We tested our framework in the two standard protocol: GTA5 to Cityscapes and SYNTHIA to Cityscapes. Code is available at: https://github.com/taveraantonio/RTDA.
GCCN: Global Context Convolutional Network
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we propose Global Context Convolutional Network (GCCN) for visual recognition. GCCN computes global features representing contextual information across image patches. These global contextual features are defined as local maxima pixels with high visual sharpness in each patch. These features are then concatenated and utilised to augment the convolutional features. The learnt feature vector is normalised using the global context features using Frobenius norm. This straightforward approach achieves high accuracy in compassion to the state-of-the-art methods with 94.6% and 95.41% on CIFAR-10 and STL-10 datasets, respectively. To explore potential impact of GCCN on other visual representation tasks, we implemented GCCN as a based model to few-shot image classification. We learn metric distances between the augmented feature vectors and their prototypes representations, similar to Prototypical and Matching Networks. GCCN outperforms state-of-the-art few-shot learning methods achieving 99.9%, 84.8% and 80.74% on Omniglot, MiniImageNet and CUB-200, respectively. GCCN has significantly improved on the accuracy of state-of-the-art prototypical and matching networks by up to 30% in different few-shot learning scenarios.
Conditional Variational Autoencoder for Learned Image Reconstruction
Venue: arXiv cs.CV 2021
Year: 2021

Learned image reconstruction techniques using deep neural networks have recently gained popularity, and have delivered promising empirical results. However, most approaches focus on one single recovery for each observation, and thus neglect the uncertainty information. In this work, we develop a novel computational framework that approximates the posterior distribution of the unknown image at each query observation. The proposed framework is very flexible: It handles implicit noise models and priors, it incorporates the data formation process (i.e., the forward operator), and the learned reconstructive properties are transferable between different datasets. Once the network is trained using the conditional variational autoencoder loss, it provides a computationally efficient sampler for the approximate posterior distribution via feed-forward propagation, and the summarizing statistics of the generated samples are used for both point-estimation and uncertainty quantification. We illustrate the proposed framework with extensive numerical experiments on positron emission tomography (with both moderate and low count levels) showing that the framework generates high-quality samples when compared with state-of-the-art methods.
Selfish & Opaque Transaction Ordering in the Bitcoin Blockchain: The Case for Chain Neutrality
Venue: arXiv cs.CR 2021
Year: 2021

Most public blockchain protocols, including the popular Bitcoin and Ethereum blockchains, do not formally specify the order in which miners should select transactions from the pool of pending (or uncommitted) transactions for inclusion in the blockchain. Over the years, informal conventions or "norms" for transaction ordering have, however, emerged via the use of shared software by miners, e.g., the GetBlockTemplate (GBT) mining protocol in Bitcoin Core. Today, a widely held view is that Bitcoin miners prioritize transactions based on their offered "transaction fee-per-byte." Bitcoin users are, consequently, encouraged to increase the fees to accelerate the commitment of their transactions, particularly during periods of congestion. In this paper, we audit the Bitcoin blockchain and present statistically significant evidence of mining pools deviating from the norms to accelerate the commitment of transactions for which they have (i) a selfish or vested interest, or (ii) received dark-fee payments via opaque (non-public) side-channels. As blockchains are increasingly being used as a record-keeping substrate for a variety of decentralized (financial technology) systems, our findings call for an urgent discussion on defining neutrality norms that miners must adhere to when ordering transactions in the chains. Finally, we make our data sets and scripts publicly available.
The Crew: The Quest for Planet Nine is NP-Complete
Venue: arXiv cs.DM 2021
Year: 2021

In this paper, we study the cooperative card game, The Crew: The Quest for Planet Nine from the viewpoint of algorithmic combinatorial game theory. The Crew: The Quest for Planet Nine, is a game based on traditional trick-taking card games, like bridge or hearts. In The Crew, players are dealt a hand of cards, with cards being from one of $c$ colors and having a value between 1 to $n$. Players also draft objectives, which correspond to a card in the current game that they must collect in order to win. Players then take turns each playing one card in a trick, with the player who played the highest value card taking the trick and all cards played in it. If all players complete all of their objectives, the players win. The game also forces players to not talk about the cards in their hand and has a number of "Task Tokens" which can modify the rules slightly. In this work, we introduce and formally define a perfect-information model of this problem, and show that the general unbounded version is computationally intractable. However, we also show that three bounded versions of this decision problem - deciding whether or not all players can complete their objectives - can be solved in polynomial time.
MOTENS: A Pedagogical Design Model for Serious Cyber Games
Venue: arXiv cs.CR 2021
Year: 2021

In the last few years, serious games have become popular, with a consensus of the benefits for teaching cyber security awareness and education. However, there is still a lack of pedagogical driven methodologies and tools to support serious games design to ensure they achieve the learning objectives. This paper proposes MOTENS, a pedagogical model, to design serious cyber games based on the gaps we identified in the current games design models and the lessons learnt from creating a serious tabletop game called Riskio, designed to teach cyber security awareness and education. The MOTENS model has six high-level components. Five components are linked to the games/design mechanics, and one component, `Theory', that supports the design's cognitive principles, including players' motivation. The model is used to design serious cyber games and goes through five stages, from identifying and segmenting target players, steps to creating game mechanics linked to pedagogy instruction and then to testing to create a serious game that is designed to achieve the games learning objectives.