Reviewerly/exHarmony huggingface dataset:
Model-Contrastive Federated Learning

Federated learning enables multiple parties to collaboratively train a machine learning model without communicating their local data. A key challenge in federated learning is to handle the heterogeneity of local data distribution across parties. Although many studies have been proposed to address this challenge, we find that they fail to achieve high performance in image datasets with deep learning models. In this paper, we propose MOON: model-contrastive federated learning. MOON is a simple and effective federated learning framework. The key idea of MOON is to utilize the similarity between model representations to correct the local training of individual parties, i.e., conducting contrastive learning in model-level. Our extensive experiments show that MOON significantly outperforms the other state-of-the-art federated learning algorithms on various image classification tasks.
LeViT: a Vision Transformer in ConvNet’s Clothing for Faster Inference

We design a family of image classification architectures that optimize the trade-off between accuracy and efficiency in a high-speed regime. Our work exploits recent findings in attention-based architectures, which are competitive on highly parallel processing hardware. We revisit principles from the extensive literature on convolutional neural networks to apply them to transformers, in particular activation maps with decreasing resolutions. We also introduce the attention bias, a new way to integrate positional information in vision transformers.As a result, we propose LeViT: a hybrid neural network for fast inference image classification. We consider different measures of efficiency on different hardware platforms, so as to best reflect a wide range of application scenarios. Our extensive experiments empirically validate our technical choices and show they are suitable to most architectures. Overall, LeViT significantly outperforms existing convnets and vision transformers with respect to the speed/accuracy tradeoff. For example, at 80% ImageNet top-1 accuracy, LeViT is 5 times faster than EfficientNet on CPU. We release the code at https://github.com/facebookresearch/LeViT.
Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Loss

In this paper we present an end-to-end speech recognition model with Transformer encoders that can be used in a streaming speech recognition system. Transformer computation blocks based on self-attention are used to encode both audio and label sequences independently. The activations from both audio and label encoders are combined with a feed-forward layer to compute a probability distribution over the label space for every combination of acoustic frame position and label history. This is similar to the Recurrent Neural Network Transducer (RNN-T) model, which uses RNNs for information encoding instead of Transformer encoders. The model is trained with the RNN-T loss well-suited to streaming decoding. We present results on the LibriSpeech dataset showing that limiting the left context for self-attention in the Transformer layers makes decoding computationally tractable for streaming, with only a slight degradation in accuracy. We also show that the full attention version of our model beats the-state-of-the art accuracy on the LibriSpeech benchmarks. Our results also show that we can bridge the gap between full attention and limited attention versions of our model by attending to a limited number of future frames.
The Cognitive Affective Model of Immersive Learning (CAMIL): a Theoretical Research-Based Model of Learning in Immersive Virtual Reality

Abstract There has been a surge in interest and implementation of immersive virtual reality (IVR)-based lessons in education and training recently, which has resulted in many studies on the topic. There are recent reviews which summarize this research, but little work has been done that synthesizes the existing findings into a theoretical framework. The Cognitive Affective Model of Immersive Learning (CAMIL) synthesizes existing immersive educational research to describe the process of learning in IVR. The general theoretical framework of the model suggests that instructional methods which are based on evidence from research with less immersive media generalize to learning in IVR. However, the CAMIL builds on evidence that media interacts with method. That is, certain methods which facilitate the affordances of IVR are specifically relevant in this medium. The CAMIL identifies presence and agency as the general psychological affordances of learning in IVR, and describes how immersion, control factors, and representational fidelity facilitate these affordances. The model describes six affective and cognitive factors that can lead to IVR-based learning outcomes including interest, motivation, self-efficacy, embodiment, cognitive load, and self-regulation. The model also describes how these factors lead to factual, conceptual, and procedural knowledge acquisition and knowledge transfer. Implications for future research and instructional design are proposed.
Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit

Application-specific optical processors have been considered disruptive technologies for modern computing that can fundamentally accelerate the development of artificial intelligence (AI) by offering substantially improved computing performance. Recent advancements in optical neural network architectures for neural information processing have been applied to perform various machine learning tasks. However, the existing architectures have limited complexity and performance; and each of them requires its own dedicated design that cannot be reconfigured to switch between different neural network models for different applications after deployment. Here, we propose an optoelectronic reconfigurable computing paradigm by constructing a diffractive processing unit (DPU) that can efficiently support different neural networks and achieve a high model complexity with millions of neurons. It allocates almost all of its computational operations optically and achieves extremely high speed of data modulation and large-scale network parameter updating by dynamically programming optical modulators and photodetectors. We demonstrated the reconfiguration of the DPU to implement various diffractive feedforward and recurrent neural networks and developed a novel adaptive training approach to circumvent the system imperfections. We applied the trained networks for high-speed classifying of handwritten digit images and human action videos over benchmark datasets, and the experimental results revealed a comparable classification accuracy to the electronic computing approaches. Furthermore, our prototype system built with off-the-shelf optoelectronic components surpasses the performance of state-of-the-art graphics processing units (GPUs) by several times on computing speed and more than an order of magnitude on system energy efficiency.
Federated learning on non-IID data: A survey

Federated learning is an emerging distributed machine learning framework for privacy preservation. However, models trained in federated learning usually have worse performance than those trained in the standard centralized learning mode, especially when the training data are not independent and identically distributed (Non-IID) on the local devices. In this survey, we provide a detailed analysis of the influence of Non-IID data on both parametric and non-parametric machine learning models in both horizontal and vertical federated learning. In addition, current research work on handling challenges of Non-IID data in federated learning are reviewed, and both advantages and disadvantages of these approaches are discussed. Finally, we suggest several future research directions before concluding the paper.
Neural Architecture Search

Deep Learning has enabled remarkable progress over the last years on a variety of tasks, such as image recognition, speech recognition, and machine translation. One crucial aspect for this progress are novel neural architectures. Currently employed architectures have mostly been developed manually by human experts, which is a time-consuming and error-prone process. Because of this, there is growing interest in automated neural architecture search methods. We provide an overview of existing work in this field of research and categorize them according to three dimensions: search space, search strategy, and performance estimation strategy.
A Blockchain-Based Smart Contract System for Healthcare Management

Blockchain is evolving to be a secure and reliable platform for secure data sharing in application areas such as the financial sector, supply chain management, food industry, energy sector, internet of things and healthcare. In this paper, we review existing literature and applications available for the healthcare system using blockchain technology. Besides, this work also proposes multiple workflows involved in the healthcare ecosystem using blockchain technology for better data management. Different medical workflows have been designed and implemented using the ethereum blockchain platform which involves complex medical procedures like surgery and clinical trials. This also includes accessing and managing a large amount of medical data. Within the implementation of the workflows of the medical smart contract system for healthcare management, the associated cost has been estimated for this system in terms of a feasibility study which has been comprehensively presented in this paper. This work would facilitate multiple stakeholders who are involved within the medical system to deliver better healthcare services and optimize cost.
Service robots, customers and service employees: what can we learn from the academic literature and where are the gaps?

Purpose Robots are predicted to have a profound impact on the service sector. The emergence of robots has attracted increasing interest from business scholars and practitioners alike. In this article, we undertake a systematic review of the business literature about the impact of service robots on customers and employees with the objective of guiding future research. Design/methodology/approach We analyzed the literature on service robots as they relate to customers and employees in business journals listed in the Financial Times top 50 journals plus all journals covered in the cross-disciplinary SERVSIG literature alerts. Findings The analysis of the identified studies yielded multiple observations about the impact of service robots on customers (e.g. overarching frameworks on acceptance and usage of service robots; characteristics of service robots and anthropomorphism; and potential for enhanced and deteriorated service experiences) and service employees (e.g. employee benefits such as reduced routine work, enhanced productivity and job satisfaction; potential negative consequences such as loss of autonomy and a range of negative psychological outcomes; opportunities for human–robot collaboration; job insecurity; and robot-related up-skilling and development requirements). We also conclude that current research on service robots is fragmented, is largely conceptual in nature and focused on the initial adoption stage. We feel that more research is needed to build an overarching theory. In addition, more empirical research is needed, especially on the long(er)-term usage service robots on actual behaviors, the well-being and potential downsides and (ethical) risks for customers and service employees. Research limitations/implications Our review focused on the business and service literature. Future work may want to include additional literature streams, including those in computer science, engineering and information systems. Originality/value This article is the first to synthesize the business and service literature on the impact of service robots on customers and employees.
Power of data in quantum machine learning

Abstract The use of quantum computing for machine learning is among the most exciting prospective applications of quantum technologies. However, machine learning tasks where data is provided can be considerably different than commonly studied computational tasks. In this work, we show that some problems that are classically hard to compute can be easily predicted by classical machines learning from data. Using rigorous prediction error bounds as a foundation, we develop a methodology for assessing potential quantum advantage in learning tasks. The bounds are tight asymptotically and empirically predictive for a wide range of learning models. These constructions explain numerical results showing that with the help of data, classical machine learning models can be competitive with quantum models even if they are tailored to quantum problems. We then propose a projected quantum model that provides a simple and rigorous quantum speed-up for a learning problem in the fault-tolerant regime. For near-term implementations, we demonstrate a significant prediction advantage over some classical models on engineered data sets designed to demonstrate a maximal quantum advantage in one of the largest numerical tests for gate-based quantum machine learning to date, up to 30 qubits.
Pre-trained models: Past, present and future

Large-scale pre-trained models (PTMs) such as BERT and GPT have recently achieved great success and become a milestone in the field of artificial intelligence (AI). Owing to sophisticated pre-training objectives and huge model parameters, large-scale PTMs can effectively capture knowledge from massive labeled and unlabeled data. By storing knowledge into huge parameters and fine-tuning on specific tasks, the rich knowledge implicitly encoded in huge parameters can benefit a variety of downstream tasks, which has been extensively demonstrated via experimental verification and empirical analysis. It is now the consensus of the AI community to adopt PTMs as backbone for downstream tasks rather than learning models from scratch. In this paper, we take a deep look into the history of pre-training, especially its special relation with transfer learning and self-supervised learning, to reveal the crucial position of PTMs in the AI development spectrum. Further, we comprehensively review the latest breakthroughs of PTMs. These breakthroughs are driven by the surge of computational power and the increasing availability of data, towards four important directions: designing effective architectures, utilizing rich contexts, improving computational efficiency, and conducting interpretation and theoretical analysis. Finally, we discuss a series of open problems and research directions of PTMs, and hope our view can inspire and advance the future study of PTMs.
TransBTS: Multimodal Brain Tumor Segmentation Using Transformer

Transformer, which can benefit from global (long-range) information modeling using self-attention mechanisms, has been successful in natural language processing and 2D image classification recently. However, both local and global features are crucial for dense prediction tasks, especially for 3D medical image segmentation. In this paper, we for the first time exploit Transformer in 3D CNN for MRI Brain Tumor Segmentation and propose a novel network named TransBTS based on the encoder-decoder structure. To capture the local 3D context information, the encoder first utilizes 3D CNN to extract the volumetric spatial feature maps. Meanwhile, the feature maps are reformed elaborately for tokens that are fed into Transformer for global feature modeling. The decoder leverages the features embedded by Transformer and performs progressive upsampling to predict the detailed segmentation map. Extensive experimental results on both BraTS 2019 and 2020 datasets show that TransBTS achieves comparable or higher results than previous state-of-the-art 3D methods for brain tumor segmentation on 3D MRI scans. The source code is available at https://github.com/Wenxuan-1119/TransBTS.
TOOD: Task-aligned One-stage Object Detection

One-stage object detection is commonly implemented by optimizing two sub-tasks: object classification and localization, using heads with two parallel branches, which might lead to a certain level of spatial misalignment in predictions between the two tasks. In this work, we propose a Task-aligned One-stage Object Detection (TOOD) that explicitly aligns the two tasks in a learning-based manner. First, we design a novel Task-aligned Head (T-Head) which offers a better balance between learning task-interactive and task-specific features, as well as a greater flexibility to learn the alignment via a task-aligned predictor. Second, we propose Task Alignment Learning (TAL) to explicitly pull closer (or even unify) the optimal anchors for the two tasks during training via a designed sample assignment scheme and a task-aligned loss. Extensive experiments are conducted on MS-COCO, where TOOD achieves a 51.1 AP at single-model single-scale testing. This surpasses the recent one-stage detectors by a large margin, such as ATSS [30] (47.7 AP), GFL [14] (48.2 AP), and PAA [9] (49.0 AP), with fewer parameters and FLOPs. Qualitative results also demonstrate the effectiveness of TOOD for better aligning the tasks of object classification and localization. Code is available at https://github.com/fcjian/TOOD.
Multi-objective multi-robot path planning in continuous environment using an enhanced genetic algorithm

This paper presents a hybrid approach for path planning of multiple mobile robots in continuous environments. For this purpose, first, an innovative Artificial Potential Field (APF) algorithm is presented to find all feasible paths between the start and destination locations in a discrete gridded environment. Next, an enhanced Genetic Algorithm (EGA) is developed to improve the initial paths in continuous space and find the optimal path between start and destination locations. The proposed APF works based on a time-efficient deterministic scheme to find a set of feasible initial paths and is guaranteed to find a feasible path if one exists. The EGA utilizes five customized crossover and mutation operators to improve the initial paths. In this paper, path length, smoothness, and safety are combined to form a multi-objective path planning problem. In addition, the proposed method is extended to deal with multiple mobile robot path planning problem. For this purpose, a new term is added to the objective function which measures the distance between robots and a collision removal operator is added to the EGA to remove possible collision between paths. To assess the efficiency of the proposed algorithm, 12 planar environments with different sizes and complexities were examined. Evaluations showed that the control parameters of the proposed algorithm do not affect the performance of the EGA considerably. Moreover, a comparative study has been made between the proposed algorithm, A*, PRM, B-RRT and Particle Swarm Optimization (PSO). The comparative study showed that the proposed algorithm outperforms PSO as well as well-recognized deterministic (A*) and probabilistic (PRM and B-RRT) path planning algorithms in terms of path length, run time, and success rate. Finally, simulations proved the efficiency of the proposed algorithm for a four-robot path planning problem. In this case, not only the proposed algorithm determined collision-free paths, but also it found near optimal solution for all robots.
Blockchain-Based Internet of Vehicles: Distributed Network Architecture and Performance Analysis

The rapid growth of Internet of Vehicles (IoV) has brought huge challenges for large data storage, intelligent management, and information security for the entire system. The traditional centralized management approach for IoV faces the difficulty in dealing with real-time response. The blockchain, as an effective technology for decentralized distributed storage and security management, has already showed great advantages in its application of Bitcoin. In this paper, we investigate how the blockchain technology could be extended to the application of vehicle networking, especially with the consideration of the distributed and secure storage of big data. We define several types of nodes such as vehicle and roadside for vehicle networks and form several sub-blockchain networks. In this paper, we present a model of the outward transmission of vehicle blockchain data, and then give detail theoretical analysis and numerical results. This paper has shown the potential to guide the application of blockchain for future vehicle networking.
Crowd Counting and Density Estimation by Trellis Encoder-Decoder Networks

Crowd counting has recently attracted increasing interest in computer vision but remains a challenging problem. In this paper, we propose a trellis encoder-decoder network (TEDnet) for crowd counting, which focuses on generating high-quality density estimation maps. The major contributions are four-fold. First, we develop a new trellis architecture that incorporates multiple decoding paths to hierarchically aggregate features at different encoding stages, which improves the representative capability of convolutional features for large variations in objects. Second, we employ dense skip connections interleaved across paths to facilitate sufficient multi-scale feature fusions, which also helps TEDnet to absorb the supervision information. Third, we propose a new combinatorial loss to enforce similarities in local coherence and spatial correlation between maps. By distributedly imposing this combinatorial loss on intermediate outputs, TEDnet can improve the back-propagation process and alleviate the gradient vanishing problem. Finally, on four widely-used benchmarks, our TEDnet achieves the best overall performance in terms of both density map quality and counting accuracy, with an improvement up to 14% in MAE metric. These results validate the effectiveness of TEDnet for crowd counting.
Multimodal Transformer With Multi-View Visual Representation for Image Captioning

Image captioning aims to automatically generate a natural language description of a given image, and most state-of-the-art models have adopted an encoder-decoder framework. The framework consists of a convolution neural network (CNN)-based image encoder that extracts region-based visual features from the input image, and an recurrent neural network (RNN) based caption decoder that generates the output caption words based on the visual features with the attention mechanism. Despite the success of existing studies, current methods only model the co-attention that characterizes the inter-modal interactions while neglecting the self-attention that characterizes the intra-modal interactions. Inspired by the success of the Transformer model in machine translation, here we extend it to a Multimodal Transformer (MT) model for image captioning. Compared to existing image captioning approaches, the MT model simultaneously captures intra- and inter-modal interactions in a unified attention block. Due to the in-depth modular composition of such attention blocks, the MT model can perform complex multimodal reasoning and output accurate captions. Moreover, to further improve the image captioning performance, multi-view visual features are seamlessly introduced into the MT model. We quantitatively and qualitatively evaluate our approach using the benchmark MSCOCO image captioning dataset and conduct extensive ablation studies to investigate the reasons behind its effectiveness. The experimental results show that our method significantly outperforms the previous state-of-the-art methods. With an ensemble of seven models, our solution ranks the 1st place on the real-time leaderboard of the MSCOCO image captioning challenge at the time of the writing of this paper.