Reviewerly/exHarmony huggingface dataset:
COVID-19 Sensing: Negative Sentiment Analysis on Social Media in China via BERT Model

Coronavirus disease 2019 (COVID-19) poses massive challenges for the world.Public sentiment analysis during the outbreak provides insightful information in making appropriate public health responses.On Sina Weibo, a popular Chinese social media, posts with negative sentiment are valuable in analyzing public concerns.999,978 randomly selected COVID-19 related Weibo posts from 1 January 2020 to 18 February 2020 are analyzed.Specifically, the unsupervised BERT (Bidirectional Encoder Representations from Transformers) model is adopted to classify sentiment categories (positive, neutral, and negative) and TF-IDF (term frequency-inverse document frequency) model is used to summarize the topics of posts.Trend analysis and thematic analysis are conducted to identify characteristics of negative sentiment.In general, the fine-tuned BERT conducts sentiment classification with considerable accuracy.Besides, topics extracted by TF-IDF precisely convey characteristics of posts regarding COVID-19.As a result, we observed that people concern four aspects regarding COVID-19, the virus Origin (Gamey Food, 3.08%; Bat, 2.70%; Conspiracy Theory, 1.43%), Symptom (Fever, 2.13%; Cough, 1.19%), Production Activity (Go to Work, 1.94%; Resume Work, 1.12%; School New Semester Beginning, 1.06%) and Public Health Control (Temperature Taking, 1.39%; Coronavirus Cover-up, 1.26%; City Shutdown, 1.09%).Results from Weibo posts provide constructive instructions on public health responses, that transparent information sharing and scientific guidance might help alleviate public concerns.
Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation

Recently, considerable effort has been devoted to deep domain adaptation in computer vision and machine learning communities. However, most of existing work only concentrates on learning shared feature representation by minimizing the distribution discrepancy across different domains. Due to the fact that all the domain alignment approaches can only reduce, but not remove the domain shift, target domain samples distributed near the edge of the clusters, or far from their corresponding class centers are easily to be misclassified by the hyperplane learned from the source domain. To alleviate this issue, we propose to joint domain alignment and discriminative feature learning, which could benefit both domain alignment and final classification. Specifically, an instance-based discriminative feature learning method and a center-based discriminative feature learning method are proposed, both of which guarantee the domain invariant features with better intra-class compactness and inter-class separability. Extensive experiments show that learning the discriminative features in the shared feature space can significantly boost the performance of deep domain adaptation methods.
Background Suppression Network for Weakly-Supervised Temporal Action Localization

Weakly-supervised temporal action localization is a very challenging problem because frame-wise labels are not given in the training stage while the only hint is video-level labels: whether each video contains action frames of interest. Previous methods aggregate frame-level class scores to produce video-level prediction and learn from video-level action labels. This formulation does not fully model the problem in that background frames are forced to be misclassified as action classes to predict video-level labels accurately. In this paper, we design Background Suppression Network (BaS-Net) which introduces an auxiliary class for background and has a two-branch weight-sharing architecture with an asymmetrical training strategy. This enables BaS-Net to suppress activations from background frames to improve localization performance. Extensive experiments demonstrate the effectiveness of BaS-Net and its superiority over the state-of-the-art methods on the most popular benchmarks – THUMOS'14 and ActivityNet. Our code and the trained model are available at https://github.com/Pilhyeon/BaSNet-pytorch.
Agnostic federated learning

A key learning scenario in large-scale applications is that of federated learning, where a centralized model is trained based on data originating from a large number of clients. We argue that, with the existing training and inference, federated models can be biased towards different clients. Instead, we propose a new framework of agnostic federated learning, where the centralized model is optimized for any target distribution formed by a mixture of the client distributions. We further show that this framework naturally yields a notion of fairness. We present data-dependent Rademacher complexity guarantees for learning with this objective, which guide the definition of an algorithm for agnostic federated learning. We also give a fast stochastic optimization algorithm for solving the corresponding optimization problem, for which we prove convergence bounds, assuming a convex loss function and hypothesis set. We further empirically demonstrate the benefits of our approach in several datasets. Beyond federated learning, our framework and algorithm can be of interest to other learning scenarios such as cloud computing, domain adaptation, drifting, and other contexts where the training and test distributions do not coincide.
PyTorch distributed

This paper presents the design, implementation, and evaluation of the PyTorch distributed data parallel module. Py-Torch is a widely-adopted scientific computing package used in deep learning research and applications. Recent advances in deep learning argue for the value of large datasets and large models, which necessitates the ability to scale out model training to more computational resources. Data parallelism has emerged as a popular solution for distributed training thanks to its straightforward principle and broad applicability. In general, the technique of distributed data parallelism replicates the model on every computational resource to generate gradients independently and then communicates those gradients at each iteration to keep model replicas consistent. Despite the conceptual simplicity of the technique, the subtle dependencies between computation and communication make it non-trivial to optimize the distributed training efficiency. As of v1.5, PyTorch natively provides several techniques to accelerate distributed data parallel, including bucketing gradients, overlapping computation with communication, and skipping gradient synchronization. Evaluations show that, when configured appropriately, the PyTorch distributed data parallel module attains near-linear scalability using 256 GPUs.
MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking

Abstract Standardized benchmarks have been crucial in pushing the performance of computer vision algorithms, especially since the advent of deep learning. Although leaderboards should not be over-claimed, they often provide the most objective measure of performance and are therefore important guides for research. We present MOTChallenge , a benchmark for single-camera Multiple Object Tracking (MOT) launched in late 2014, to collect existing and new data and create a framework for the standardized evaluation of multiple object tracking methods. The benchmark is focused on multiple people tracking, since pedestrians are by far the most studied object in the tracking community, with applications ranging from robot navigation to self-driving cars. This paper collects the first three releases of the benchmark: (i) MOT15 , along with numerous state-of-the-art results that were submitted in the last years, (ii) MOT16 , which contains new challenging videos, and (iii) MOT17 , that extends MOT16 sequences with more precise labels and evaluates tracking performance on three different object detectors. The second and third release not only offers a significant increase in the number of labeled boxes, but also provide labels for multiple object classes beside pedestrians, as well as the level of visibility for every single object of interest. We finally provide a categorization of state-of-the-art trackers and a broad error analysis. This will help newcomers understand the related work and research trends in the MOT community, and hopefully shed some light into potential future research directions.
Shakedrop Regularization for Deep Residual Learning

Overfitting is a crucial problem in deep neural networks, even in the latest network architectures. In this paper, to relieve the overfitting effect of ResNet and its improvements (i.e., Wide ResNet, PyramidNet, and ResNeXt), we propose a new regularization method called ShakeDrop regularization. ShakeDrop is inspired by Shake-Shake, which is an effective regularization method, but can be applied to ResNeXt only. ShakeDrop is more effective than Shake-Shake and can be applied not only to ResNeXt but also ResNet, Wide ResNet, and PyramidNet. An important key is to achieve stability of training. Because effective regularization often causes unstable training, we introduce a training stabilizer, which is an unusual use of an existing regularizer. Through experiments under various conditions, we demonstrate the conditions under which ShakeDrop works well.
Cryptocurrency trading: a comprehensive survey

Abstract In recent years, the tendency of the number of financial institutions to include cryptocurrencies in their portfolios has accelerated. Cryptocurrencies are the first pure digital assets to be included by asset managers. Although they have some commonalities with more traditional assets, they have their own separate nature and their behaviour as an asset is still in the process of being understood. It is therefore important to summarise existing research papers and results on cryptocurrency trading, including available trading platforms, trading signals, trading strategy research and risk management. This paper provides a comprehensive survey of cryptocurrency trading research, by covering 146 research papers on various aspects of cryptocurrency trading ( e . g ., cryptocurrency trading systems, bubble and extreme condition, prediction of volatility and return, crypto-assets portfolio construction and crypto-assets, technical trading and others). This paper also analyses datasets, research trends and distribution among research objects (contents/properties) and technologies, concluding with some promising opportunities that remain open in cryptocurrency trading.
Exploration, Inference, and Prediction in Neuroscience and Biomedicine

Recent decades have seen dramatic progress in brain research. These advances were often buttressed by probing single variables to make circumscribed discoveries, typically through null hypothesis significance testing. New ways for generating massive data fueled tension between the traditional methodology that is used to infer statistically relevant effects in carefully chosen variables, and pattern-learning algorithms that are used to identify predictive signatures by searching through abundant information. In this article we detail the antagonistic philosophies behind two quantitative approaches: certifying robust effects in understandable variables, and evaluating how accurately a built model can forecast future outcomes. We discourage choosing analytical tools via categories such as 'statistics' or 'machine learning'. Instead, to establish reproducible knowledge about the brain, we advocate prioritizing tools in view of the core motivation of each quantitative analysis: aiming towards mechanistic insight or optimizing predictive accuracy.
On the interpretability of machine learning-based model for predicting hypertension

Although complex machine learning models are commonly outperforming the traditional simple interpretable models, clinicians find it hard to understand and trust these complex models due to the lack of intuition and explanation of their predictions. The aim of this study to demonstrate the utility of various model-agnostic explanation techniques of machine learning models with a case study for analyzing the outcomes of the machine learning random forest model for predicting the individuals at risk of developing hypertension based on cardiorespiratory fitness data. The dataset used in this study contains information of 23,095 patients who underwent clinician-referred exercise treadmill stress testing at Henry Ford Health Systems between 1991 and 2009 and had a complete 10-year follow-up. Five global interpretability techniques (Feature Importance, Partial Dependence Plot, Individual Conditional Expectation, Feature Interaction, Global Surrogate Models) and two local interpretability techniques (Local Surrogate Models, Shapley Value) have been applied to present the role of the interpretability techniques on assisting the clinical staff to get better understanding and more trust of the outcomes of the machine learning-based predictions. Several experiments have been conducted and reported. The results show that different interpretability techniques can shed light on different insights on the model behavior where global interpretations can enable clinicians to understand the entire conditional distribution modeled by the trained response function. In contrast, local interpretations promote the understanding of small parts of the conditional distribution for specific instances. Various interpretability techniques can vary in their explanations for the behavior of the machine learning model. The global interpretability techniques have the advantage that it can generalize over the entire population while local interpretability techniques focus on giving explanations at the level of instances. Both methods can be equally valid depending on the application need. Both methods are effective methods for assisting clinicians on the medical decision process, however, the clinicians will always remain to hold the final say on accepting or rejecting the outcome of the machine learning models and their explanations based on their domain expertise.
Deep learning technology for improving cancer care in society: New directions in cancer imaging driven by artificial intelligence

The goal of this study is to show emerging applications of deep learning technology in cancer imaging. Deep learning technology is a family of computational methods that allow an algorithm to program itself by learning from a large set of examples that demonstrate the desired behavior. Applications of deep learning technology to cancer imaging can assist pathologists in the detection and classification of cancer in the early stages of its development to allow patients to have appropriate treatments that can increase their survival. Statistical analyses and other analytical approaches, based on data of ScienceDirect (a source for scientific research), suggest that the sharp increase of the studies of deep learning technology in cancer imaging seems to be driven by high rates of mortality of some types of cancer (e.g., lung and breast) in order to solve consequential problems of a more accurate detection and characterization of cancer types to apply efficient anti-cancer therapies. Moreover, this study also shows sources of the trajectories of deep learning technology in cancer imaging at level of scientific subject areas, universities and countries with the highest scientific production in these research fields. This new technology, in accordance with Amara's law, can generate a shift of technological paradigm for diagnostic assessment of any cancer type and disease. This new technology can also generate socioeconomic benefits for poor regions because they can send digital images to labs of other developed regions to have diagnosis of cancer types, reducing as far as possible current gap in healthcare sector among different regions.
Tree-CNN: A hierarchical Deep Convolutional Neural Network for incremental learning

Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, such as “catastrophic forgetting”, and sensitivity to hyper-parameter tuning. However, in this modern world, data is constantly evolving, and our deep learning models are required to adapt to these changes. In this paper, we propose an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available. The network grows in a tree-like fashion to accommodate new classes of data, while preserving the ability to distinguish the previously trained classes. The network organizes the incrementally available data into feature-driven super-classes and improves upon existing hierarchical CNN models by adding the capability of self-growth. The proposed hierarchical model, when compared against fine-tuning a deep network, achieves significant reduction of training effort, while maintaining competitive accuracy on CIFAR-10 and CIFAR-100.
Electroencephalograms during Mental Arithmetic Task Performance

This work has been carried out to support the investigation of the electroencephalogram (EEG) Fourier power spectral, coherence, and detrended fluctuation characteristics during performance of mental tasks. To this aim, the presented dataset contains International 10/20 system EEG recordings from subjects under mental cognitive workload (performing mental serial subtraction) and the corresponding reference background EEGs. Based on the subtraction task performance (number of subtractions and accuracy of the result), the subjects were divided into good counters and bad counters (for whom the mental task required excessive efforts). The data was recorded from 36 healthy volunteers of matched age, all of whom are students of Educational and Scientific Centre “Institute of Biology and Medicine”, National Taras Shevchenko University of Kyiv (Ukraine); the recordings are available through Physiobank platform. The dataset can be used by the neuroscience research community studying brain dynamics during cognitive workload.
Speech Model Pre-Training for End-to-End Spoken Language Understanding

Whereas conventional spoken language understanding (SLU) systems map speech to text, and then text to intent, end-to-end SLU systems map speech directly to intent through a single trainable model. Achieving high accuracy with these end-to-end models without a large amount of training data is difficult. We propose a method to reduce the data requirements of end-to-end SLU in which the model is first pre-trained to predict words and phonemes, thus learning good features for SLU. We introduce a new SLU dataset, Fluent Speech Commands, and show that our method improves performance both when the full dataset is used for training and when only a small subset is used. We also describe preliminary experiments to gauge the model's ability to generalize to new phrases not heard during training.
Attention Gate ResU-Net for Automatic MRI Brain Tumor Segmentation

Brain tumor segmentation technology plays a pivotal role in the process of diagnosis and treatment of MRI brain tumors. It helps doctors to locate and measure tumors, as well as develop treatment and rehabilitation strategies. Recently, MRI brain tumor segmentation methods based on U-Net architecture have become popular as they largely improve the segmentation accuracy by applying skip connection to combine high-level feature information and low-level feature information. Meanwhile, researchers have demonstrated that introducing attention mechanism into U-Net can enhance local feature expression and improve the performance of medical image segmentation. In this work, we aim to explore the effectiveness of a recent attention module called attention gate for brain tumor segmentation task, and a novel Attention Gate Residual U-Net model, i.e., AGResU-Net, is further presented. AGResU-Net integrates residual modules and attention gates with a primeval and single U-Net architecture, in which a series of attention gate units are added into the skip connection for highlighting salient feature information while disambiguating irrelevant and noisy feature responses. AGResU-Net not only extracts abundant semantic information to enhance the ability of feature learning, but also pays attention to the information of small-scale brain tumors. We extensively evaluate attention gate units on three authoritative MRI brain tumor benchmarks, i.e., BraTS 2017, BraTS 2018 and BraTS 2019. Experimental results illuminate that models with attention gate units, i.e., Attention Gate U-Net (AGU-Net) and AGResU-Net, outperform their baselines of U-Net and ResU-Net, respectively. In addition, AGResU-Net achieves competitive performance than the representative brain tumor segmentation methods.
Multi-Label Learning by Exploiting Label Correlations Locally

It is well known that exploiting label correlations is important for multi-label learning. Existing approaches typically exploit label correlations globally, by assuming that the label correlations are shared by all the instances. In real-world tasks, however, different instances may share different label correlations, and few correlations are globally applicable. In this paper, we propose the ML-LOC approach which allows label correlations to be exploited locally. To encode the local influence of label correlations, we derive a LOC code to enhance the feature representation of each instance. The global discrimination fitting and local correlation sensitivity are incorporated into a unified framework, and an alternating solution is developed for the optimization. Experimental results on a number of image, text and gene data sets validate the effectiveness of our approach.
Integrating emerging technologies into COVID-19 contact tracing: Opportunities, challenges and pitfalls

With no approved vaccines for treating COVID-19 as of August 2020, many health systems and governments rely on contact tracing as one of the prevention and containment methods. However, there have been instances when the infected person forgets his/her contact-persons and does not have their contact details. Therefore, this study aimed at analyzing possible opportunities and challenges of integrating emerging technologies into COVID-19 contact tracing. The study applied literature search from Google Scholar, Science Direct, PubMed, Web of Science, IEEE and WHO COVID-19 reports and guidelines analyzed. While the integration of technology-based contact tracing applications to combat COVID-19 and break transmission chains promise to yield better results, these technologies face challenges such as technical limitations, dealing with asymptomatic individuals, lack of supporting ICT infrastructure and electronic health policy, socio-economic inequalities, deactivation of mobile devices’ WIFI, GPS services, interoperability and standardization issues, security risks, privacy issues, political and structural responses, ethical and legal risks, consent and voluntariness, abuse of contact tracing apps, and discrimination. Integrating emerging technologies into COVID-19 contact tracing is seen as a viable option that policymakers, health practitioners and IT technocrats need to seriously consider in mitigating the spread of coronavirus. Further research is also required on how best to improve efficiency and effectiveness in the utilisation of emerging technologies in contact tracing while observing the security and privacy of people in fighting the COVID-19 pandemic.