Reviewerly/exHarmony huggingface dataset:
Pose-Aware Multi-Level Feature Network for Human Object Interaction Detection

Reasoning human object interactions is a core problem in human-centric scene understanding and detecting such relations poses a unique challenge to vision systems due to large variations in human-object configurations, multiple co-occurring relation instances and subtle visual difference between relation categories. To address those challenges, we propose a multi-level relation detection strategy that utilizes human pose cues to capture global spatial configurations of relations and as an attention mechanism to dynamically zoom into relevant regions at human part level. We develop a multi-branch deep network to learn a pose-augmented relation representation at three semantic levels, incorporating interaction context, object features and detailed semantic part cues. As a result, our approach is capable of generating robust predictions on fine-grained human object interactions with interpretable outputs. Extensive experimental evaluations on public benchmarks show that our model outperforms prior methods by a considerable margin, demonstrating its efficacy in handling complex scenes.
Immersive virtual reality in K‐12 and higher education: A 10‐year systematic review of empirical research

Abstract Immersive Virtual Reality (IVR) systems offer several learning affordances, that is, characteristics that can elicit learning behaviors promoting learning outcomes in educational contexts. In particular, the immersive and interactive properties of these systems represent a strong opportunity to support the effective learning experiences. The present review aims at mapping IVR systems’ use in K‐12 and higher educational contexts and investigating their effectiveness in facilitating learning in terms of knowledge attainment and retention and motivational outcomes. The synthesis of the reviewed studies ( n = 18) show that IVR can support a number of activities and experiences that in turn improve learning and motivate students to fulfill educational goals by eliciting their interest and engagement with the learning materials. The main advantage of IVR seems related to the possibility for users to have first‐hand experiences that would not be possible in the real world, simultaneously offering unique opportunities for experiential and situated learning, as well as promoting students’ motivation and engagement. Despite the promising findings, the reviewed studies present several methodological flaws: non‐randomized trials, small sample sizes, unbalanced samples and non‐validated measures. This in turn makes it difficult to generalize from these results. Future research is needed to investigate the unique opportunities of IVR in education. Practitioner Notes What is already known about this topic? There is an increasing use of immersive virtual reality (IVR) in different fields, including education. Experiences in IVR promote sense of presence, sense of embodiment, while supporting attentional and emotional engagement. What this paper adds? The paper outlines the learning affordances of IVR both at cognitive and affective level. The results of our systematic review show that IVR is a promising technology that can positively influence the students’ learning achievements and motivation to learn. The paper presents also a critical analysis of the included papers, which results in recommendation for future research and practice. Implication for practice and/or policy. The paper makes evidence‐based recommendations for teachers as well as researchers in the field. It introduces affordable and easy‐to‐use devices that meet the affordances of IVR for improving learning.
BEPO: A novel binary emperor penguin optimizer for automatic feature selection

Emperor Penguin Optimizer (EPO) is a metaheuristic algorithm which is recently developed and illustrates the emperor penguin’s huddling behaviour. However, the original version of the EPO will fix issues that are continuing in fact but not discrete. The eight separate EPO variants have been provided in this article. Four transfer features, s-shaped and v-shaped, that are used in order to map the search space into a separate research space are considered in the proposed algorithm. The output of the proposed algorithm is validated using 25 standard benchmark functions. It also analyses the statistical sense of the proposed algorithm. Experimental findings and comparisons suggest that the proposed algorithm performs better than other algorithms. The solution also applies to the issue of feature selection. The findings reveal the supremacy of the binary emperor penguin optimization algorithm.
DetCo: Unsupervised Contrastive Learning for Object Detection

We present DetCo, a simple yet effective self-supervised approach for object detection. Unsupervised pre-training methods have been recently designed for object detection, but they are usually deficient in image classification, or the opposite. Unlike them, DetCo transfers well on downstream instance-level dense prediction tasks, while maintaining competitive image-level classification accuracy. The advantages are derived from (1) multi-level supervision to intermediate representations, (2) contrastive learning between global image and local patches. These two designs facilitate discriminative and consistent global and local representation at each level of feature pyramid, improving detection and classification, simultaneously.Extensive experiments on VOC, COCO, Cityscapes, and ImageNet demonstrate that DetCo not only outperforms recent methods on a series of 2D and 3D instance-level detection tasks, but also competitive on image classification. For example, on ImageNet classification, DetCo is 6.9% and 5.0% top-1 accuracy better than InsLoc and DenseCL, which are two contemporary works designed for object detection. Moreover, on COCO detection, DetCo is 6.9 AP better than SwAV with Mask R-CNN C4. Notably, DetCo largely boosts up Sparse R-CNN, a recent strong detector, from 45.0 AP to 46.5 AP (+1.5 AP), establishing a new SOTA on COCO.
Federated Learning for Keyword Spotting

We propose a practical approach based on federated learning to solve out-of-domain issues with continuously running embedded speech-based models such as wake word detectors. We conduct an extensive empirical study of the federated averaging algorithm for the "Hey Snips" wake word based on a crowdsourced dataset that mimics a federation of wake word users. We empirically demonstrate that using an adaptive averaging strategy inspired from Adam in place of standard weighted model averaging highly reduces the number of communication rounds required to reach our target performance. The associated upstream communication costs per user are estimated at 8 MB, which is a reasonable in the context of smart home voice assistants. Additionally, the dataset used for these experiments is being open sourced with the aim of fostering further transparent research in the application of federated learning to speech data.
Management of immersive heritage tourism experiences: A conceptual model

There is potential for immersive technology, such as augmented and virtual reality, to create memorable tourism experiences, specifically for heritage tourism. However, there is a lack of conceptual clarity surrounding the management of heritage for memorable tourism experiences. Subsequently, this research note proposes a four-stage conceptual model of heritage preservation for managing heritage into digital tourism experiences. The four stages include the presentation of historical facts; contested heritage; integration of historical facts and contested heritage; and/or an alternate scenario. This research note demonstrates that integrating history with cutting-edge technology in immersive environments has the potential to not only preserve and manage heritage but to enrich the visitor experience and subsequent engagement with history.
Generation and sampling of quantum states of light in a silicon chip

Implementing large instances of quantum algorithms requires the processing of many quantum information carriers in a hardware platform that supports the integration of different components. While established semiconductor fabrication processes can integrate many photonic components, the generation and algorithmic processing of many photons has been a bottleneck in integrated photonics. Here we report the on-chip generation and processing of quantum states of light with up to eight photons in quantum sampling algorithms. Switching between different optical pumping regimes, we implement the Scattershot, Gaussian and standard boson sampling protocols in the same silicon chip, which integrates linear and nonlinear photonic circuitry. We use these results to benchmark a quantum algorithm for calculating molecular vibronic spectra. Our techniques can be readily scaled for the on-chip implementation of specialised quantum algorithms with tens of photons, pointing the way to efficiency advantages over conventional computers.
Automated diagnosis of breast ultrasonography images using deep neural networks

Ultrasonography images of breast mass aid in the detection and diagnosis of breast cancer. Manually analyzing ultrasonography images is time-consuming, exhausting and subjective. Automated analyzing such images is desired. In this study, we develop an automated breast cancer diagnosis model for ultrasonography images. Traditional methods of automated ultrasonography images analysis employ hand-crafted features to classify images, and lack robustness to the variation in the shapes, size and texture of breast lesions, leading to low sensitivity in clinical applications. To overcome these shortcomings, we propose a method to diagnose breast ultrasonography images using deep convolutional neural networks with multi-scale kernels and skip connections. Our method consists of two components: the first one is to determine whether there are malignant tumors in the image, and the second one is to recognize solid nodules. In order to let the two networks work in a collaborative way, a region enhance mechanism based on class activation maps is proposed. The mechanism helps to improve classification accuracy and sensitivity for both networks. A cross training algorithm is introduced to train the networks. We construct a large annotated dataset containing a total of 8145 breast ultrasonography images to train and evaluate the models. All of the annotations are proven by pathological records. The proposed method is compared with two state-of-the-art approaches, and outperforms both of them by a large margin. Experimental results show that our approach achieves a performance comparable to human sonographers and can be applied to clinical scenarios.
Communication Breakdowns Between Families and Alexa

We investigate how families repair communication breakdowns with digital home assistants. We recruited 10 diverse families to use an Amazon Echo Dot in their homes for four weeks. All families had at least one child between four and 17 years old. Each family participated in pre- and post- deployment interviews. Their interactions with the Echo Dot (Alexa) were audio recorded throughout the study. We analyzed 59 communication breakdown interactions between family members and Alexa, framing our analysis with concepts from HCI and speech-language pathology. Our findings indicate that family members collaborate using discourse scaffolding (supportive communication guidance) and a variety of speech and language modifications in their attempts to repair communication breakdowns with Alexa. Alexa's responses also influence the repair strategies that families use. Designers can relieve the communication repair burden that primarily rests with families by increasing digital home assistants' abilities to collaborate together with users to repair communication breakdowns.
NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm

Deep neural networks (DNNs) have begun to have a pervasive impact on various applications of machine learning. However, the problem of finding an optimal DNN architecture for large applications is challenging. Common approaches go for deeper and larger DNN architectures but may incur substantial redundancy. To address these problems, we introduce a network growth algorithm that complements network pruning to learn both weights and compact DNN architectures during training. We propose a DNN synthesis tool (NeST) that combines both methods to automate the generation of compact and accurate DNNs. NeST starts with a randomly initialized sparse network called the seed architecture. It iteratively tunes the architecture with gradient-based growth and magnitude-based pruning of neurons and connections. Our experimental results show that NeST yields accurate, yet very compact DNNs, with a wide range of seed architecture selection. For the LeNet-300-100 (LeNet-5) architecture, we reduce network parameters by 70.2× (74.3×) and floating-point operations (FLOPs) by 79.4× (43.7×). For the AlexNet, VGG-16, and ResNet-50 architectures, we reduce network parameters (FLOPs) by 15.7× (4.6×), 33.2× (8.9×), and 4.1× (2.1×) respectively. NeST's grow-and-prune paradigm delivers significant additional parameter and FLOPs reduction relative to pruning-only methods.
Complexer-YOLO: Real-Time 3D Object Detection and Tracking on Semantic Point Clouds

Accurate detection of 3D objects is a fundamental problem in computer vision and has an enormous impact on autonomous cars, augmented/virtual reality and many applications in robotics. In this work we present a novel fusion of neural network based state-of-the-art 3D detector and visual semantic segmentation in the context of autonomous driving. Additionally, we introduce Scale-Rotation-Translation score (SRTs), a fast and highly parameterizable evaluation metric for comparison of object detections, which speeds up our inference time up to 20% and halves training time. On top, we apply state-of-the-art online multi target feature tracking on the object measurements to further increase accuracy and robustness utilizing temporal information. Our experiments on KITTI show that we achieve same results as state-of-the-art in all related categories, while maintaining the performance and accuracy trade-off and still run in real-time. Furthermore, our model is the first one that fuses visual semantic with 3D object detection.
Enhancing hospitality experience with service robots: the mediating role of rapport building

This study investigated the influence of service robot attributes on customers’ hospitality experience from the perspective of relationship building. Through literature review and a preliminary study with in-depth interviews, a conceptual framework was developed. A scenario-based experiment and questionnaire survey were designed to test the model. The results indicate that robots’ being perceived as humanlike or intelligent positively affects customer-robot rapport building and the hospitality experience. Additionally, customer-employee rapport building was found to mediate the relationship between robot attributes and the hospitality experience, but customer-robot rapport building was not. Based on these findings, theoretical contributions and practical implications were discussed.
The adverse effects of code duplication in machine learning models of code

The field of big code relies on mining large corpora of code to perform some learning task towards creating better tools for software engineers. A significant threat to this approach was recently identified by Lopes et al. (2017) who found a large amount of near-duplicate code on GitHub. However, the impact of code duplication has not been noticed by researchers devising machine learning models for source code. In this work, we explore the effects of code duplication on machine learning models showing that reported performance metrics are sometimes inflated by up to 100% when testing on duplicated code corpora compared to the performance on de-duplicated corpora which more accurately represent how machine learning models of code are used by software engineers. We present a duplication index for widely used datasets, list best practices for collecting code corpora and evaluating machine learning models on them. Finally, we release tools to help the community avoid this problem in future research.
An intelligent blockchain-based system for safe vaccine supply and supervision

Immunization is an indispensable mechanism for preventing infectious diseases in modern society, and vaccine safety is closely related to public health and national security. However, issues such as vaccine expiration and vaccine record fraud are still widespread in vaccine supply chains. Therefore, an effective management system for the supervision of vaccine supply chains is urgently required. As the next generation of core technology after the Internet, blockchain is designed to build trust mechanisms that can change current information management methods. Meanwhile, the development of machine learning technologies provides additional ways to analyze the data in information management systems. The main objective of this study is to develop a “vaccine blockchain” system based on blockchain and machine learning technologies. This vaccine blockchain system is designed to support vaccine traceability and smart contract functions, and can be used to address the problems of vaccine expiration and vaccine record fraud. Additionally, the use of machine learning models can provide valuable recommendations to immunization practitioners and recipients, allowing them to choose better immunization methods and vaccines.
Resolving data sparsity and cold start problem in collaborative filtering recommender system using Linked Open Data

The web contains a huge volume of data, and it's populating every moment to the point that human beings cannot deal with the vast amount of data manually or via traditional tools. Hence an advanced tool is required to filter such massive data and mine the valuable information. Recommender systems are among the most excellent tools for such a purpose in which collaborative filtering is widely used. Collaborative filtering (CF) has been extensively utilized to offer personalized recommendations in electronic business and social network websites. In that, matrix factorization is an efficient technique; however, it depends on past transactions of the users. Hence, there will be a data sparsity problem. Another issue with the collaborative filtering method is the cold start issue, which is due to the deficient information about new entities. A novel method is proposed to overcome the data sparsity and the cold start problem in CF. For cold start issue, Recommender System with Linked Open Data (RS-LOD) model is designed and for data sparsity problem, Matrix Factorization model with Linked Open Data is developed (MF-LOD). A LOD knowledge base “DBpedia” is used to find enough information about new entities for a cold start issue, and an improvement is made on the matrix factorization model to handle data sparsity. Experiments were done on Netflix and MovieLens datasets show that our proposed techniques are superior to other existing methods, which mean recommendation accuracy is improved.
Quartznet: Deep Automatic Speech Recognition with 1D Time-Channel Separable Convolutions

We propose a new end-to-end neural acoustic model for automatic speech recognition. The model is composed of multiple blocks with residual connections between them. Each block consists of one or more modules with 1D time-channel separable convolutional layers, batch normalization, and ReLU layers. It is trained with CTC loss. The proposed network achieves near state-of-the-art accuracy on LibriSpeech and Wall Street Journal, while having fewer parameters than all competing models. We also demonstrate that this model can be effectively fine-tuned on new datasets.
Text‐based emotion detection: Advances, challenges, and opportunities

Abstract Emotion detection (ED) is a branch of sentiment analysis that deals with the extraction and analysis of emotions. The evolution of Web 2.0 has put text mining and analysis at the frontiers of organizational success. It helps service providers provide tailor‐made services to their customers. Numerous studies are being carried out in the area of text mining and analysis due to the ease in sourcing for data and the vast benefits its deliverable offers. This article surveys the concept of ED from texts and highlights the main approaches adopted by researchers in the design of text‐based ED systems. The article further discusses some recent state‐of‐the‐art proposals in the field. The proposals are discussed in relation to their major contributions, approaches employed, datasets used, results obtained, strengths, and their weaknesses. Also, emotion‐labeled data sources are presented to provide neophytes with eligible text datasets for ED. Finally, the article presents some open issues and future research direction for text‐based ED.
Artificial Intelligence in the Agri-Food System: Rethinking Sustainable Business Models in the COVID-19 Scenario

The aim of the paper is to investigate the artificial intelligence (AI) function in agri-food industry, as well as the role of stakeholders in its supply chain. Above all, from the beginning of the new millennium, scholars and practitioners have paid an increasing attention to artificial intelligence (AI) technologies in operational processes management and challenges for new business models, in a sustainable and socially responsible perspective. Thus, the stakeholders can assume a proactive or marginal role in the value creation for business, according to their own environmental awareness. These issues appear still “open” in some industries, such as the agri-food system, where the adoption of new technologies requires rethinking and redesigning the whole business model. Methodologically, we brought forward an in-depth review of the literature about major articles in this field. Especially, the study has been conducted following two phases: firstly, we extracted from scientific databases (Web of Science, Scopus, and Google Scholar) and studied relevant articles; secondly, we analyzed the selected articles. The findings highlight interesting issues about AI towards a “space economy” to achieve sustainable and responsible business models, also in the perspective of the COVID-19 pandemic scenario. Theoretical and managerial implications are discussed.
Listen to Look: Action Recognition by Previewing Audio

In the face of the video data deluge, today's expensive clip-level classifiers are increasingly impractical. We propose a framework for efficient action recognition in untrimmed video that uses audio as a preview mechanism to eliminate both short-term and long-term visual redundancies. First, we devise an ImgAud2Vid framework that hallucinates clip-level features by distilling from lighter modalities---a single frame and its accompanying audio---reducing short-term temporal redundancy for efficient clip-level recognition. Second, building on ImgAud2Vid, we further propose ImgAud-Skimming, an attention-based long short-term memory network that iteratively selects useful moments in untrimmed videos, reducing long-term temporal redundancy for efficient video-level recognition. Extensive experiments on four action recognition datasets demonstrate that our method achieves the state-of-the-art in terms of both recognition accuracy and speed.