Reviewerly/exHarmony huggingface dataset:
Profit Allocation for Federated Learning

Due to stricter data management regulations such as General Data Protection Regulation (GDPR), traditional production mode of machine learning services is shifting to federated learning, a paradigm that allows multiple data providers to train a joint model collaboratively with their data kept locally. A key enabler for practical adoption of federated learning is how to allocate the prolit earned by the joint model to each data provider. For fair prolit allocation, a metric to quantify the contribution of each data provider to the joint model is essential. Shapley value is a classical concept in cooperative game theory which assigns a unique distribution (among the players) of a total surplus generated by the coalition of all players and has been used for data valuation in machine learning services. However, prior Shapley value based data valuation schemes either do not apply to federated learning or involve extra model training which leads to high cost. In this paper, given n data providers with data sets D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sub> , D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sub> , ⋯, D <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sub> , a federated learning algorithm A and a standard test set T, we propose the contribution index, a new Shapley value based metric lit for assessing the contribution of each data provider for the joint model trained by federated learning. The contribution index shares the same properties as Shapley value. However, direct calculation of the contribution index is time consuming, since a large number of joint models with different combinations of data sets are required to be trained and evaluated. To solve this problem, we propose two gradient based methods. The idea is to reconstruct approximately the models on different combinations of the data sets through the intermediate results of the training process of federated learning so as to avoid extra training. The lirst method reconstructs models by updating the initial global model in federated learning with the gradients in different rounds. Then it calculates the contribution index by the performance of these reconstructed models. The second method calculates contribution index in each round by updating the global model in the previous round with the gradients in the current round. Contribution indexes of multiple rounds are then added with elaborated weights to get the linal result. We conduct extensive experiments on the MNIST data set in different settings. The results demonstrate that the proposed methods can approximate the exact contribution index effectively and achieve a time speed up of up to 2x-100x compared with the exact calculation and other baselines extended from existing work.
AUTo<i>Sen</i>: Deep-Learning-Based Implicit Continuous Authentication Using Smartphone Sensors

Smartphones have become crucial for our daily life activities and are increasingly loaded with our personal information to perform several sensitive tasks, including, mobile banking and communication, and are used for storing private photos and files. Therefore, there is a high demand for applying usable authentication techniques that prevent unauthorized access to sensitive information. In this article, we propose AUToSen, a deep-learning-based active authentication approach that exploits sensors in consumer-grade smartphones to authenticate a user. Unlike conventional approaches, AUToSen is based on deep learning to identify user distinct behavior from the embedded sensors with and without the user's interaction with the smartphone. We investigate different deep learning architectures in modeling and capturing users' behavioral patterns for the purpose of authentication. Moreover, we explore the sufficiency of sensory data required to accurately authenticate users. We evaluate AUToSen on a real-world data set that includes sensors data of 84 participants' smartphones collected using our designed data-collection application. The experiments show that AUToSen operates accurately using readings of only three sensors (accelerometer, gyroscope, and magnetometer) with a high authentication frequency, e.g., one authentication attempt every 0.5 s. Using sensory data of one second enables an authentication F1-score of approximately 98%, false acceptance rate (FAR) of 0.95%, false rejection rate (FRR) of 6.67%, and equal error rate (EER) of 0.41%. While using sensory data of half a second enables an authentication F1-score of 97.52%, FAR of 0.96%, FRR of 8.08%, and EER of 0.09%. Moreover, we investigate the effects of using different sensory data at variable sampling periods on the performance of the authentication models under various settings and learning architectures.
Multi-Agent Reinforcement Learning Based Frame Sampling for Effective Untrimmed Video Recognition

Video Recognition has drawn great research interest and great progress has been made. A suitable frame sampling strategy can improve the accuracy and efficiency of recognition. However, mainstream solutions generally adopt hand-crafted frame sampling strategies for recognition. It could degrade the performance, especially in untrimmed videos, due to the variation of frame-level saliency. To this end, we concentrate on improving untrimmed video classification via developing a learning-based frame sampling strategy. We intuitively formulate the frame sampling procedure as multiple parallel Markov decision processes, each of which aims at picking out a frame/clip by gradually adjusting an initial sampling. Then we propose to solve the problems with multi-agent reinforcement learning (MARL). Our MARL framework is composed of a novel RNN-based context-aware observation network which jointly models context information among nearby agents and historical states of a specific agent, a policy network which generates the probability distribution over a predefined action space at each step and a classification network for reward calculation as well as final recognition. Extensive experimental results show that our MARL-based scheme remarkably outperforms hand-crafted strategies with various 2D and 3D baseline methods. Our single RGB model achieves a comparable performance of ActivityNet v1.3 champion submission with multi-modal multi-model fusion and new state-of-the-art results on YouTube Birds and YouTube Cars.
Algorithmic Effects on the Diversity of Consumption on Spotify

On many online platforms, users can engage with millions of pieces of content, which they discover either organically or through algorithmically-generated recommendations. While the short-term benefits of recommender systems are well-known, their long-term impacts are less well understood. In this work, we study the user experience on Spotify, a popular music streaming service, through the lens of diversity—the coherence of the set of songs a user listens to. We use a high-fidelity embedding of millions of songs based on listening behavior on Spotify to quantify how musically diverse every user is, and find that high consumption diversity is strongly associated with important long-term user metrics, such as conversion and retention. However, we also find that algorithmically-driven listening through recommendations is associated with reduced consumption diversity. Furthermore, we observe that when users become more diverse in their listening over time, they do so by shifting away from algorithmic consumption and increasing their organic consumption. Finally, we deploy a randomized experiment and show that algorithmic recommendations are more effective for users with lower diversity. Our work illuminates a central tension in online platforms: how do we recommend content that users are likely to enjoy in the short term while simultaneously ensuring they can remain diverse in their consumption in the long term?
Visual Search: How Do We Find What We Are Looking For?

In visual search tasks, observers look for targets among distractors. In the lab, this often takes the form of multiple searches for a simple shape that may or may not be present among other items scattered at random on a computer screen (e.g., Find a red T among other letters that are either black or red.). In the real world, observers may search for multiple classes of target in complex scenes that occur only once (e.g., As I emerge from the subway, can I find lunch, my friend, and a street sign in the scene before me?). This article reviews work on how search is guided intelligently. I ask how serial and parallel processes collaborate in visual search, describe the distinction between search templates in working memory and target templates in long-term memory, and consider how searches are terminated.
An End-to-End Learning Framework for Video Compression

Traditional video compression approaches build upon the hybrid coding framework with motion-compensated prediction and residual transform coding. In this paper, we propose the first end-to-end deep video compression framework to take advantage of both the classical compression architecture and the powerful non-linear representation ability of neural networks. Our framework employs pixel-wise motion information, which is learned from an optical flow network and further compressed by an auto-encoder network to save bits. The other compression components are also implemented by the well-designed networks for high efficiency. All the modules are jointly optimized by using the rate-distortion trade-off and can collaborate with each other. More importantly, the proposed deep video compression framework is very flexible and can be easily extended by using lightweight or advanced networks for higher speed or better efficiency. We also propose to introduce the adaptive quantization layer to reduce the number of parameters for variable bitrate coding. Comprehensive experimental results demonstrate the effectiveness of the proposed framework on the benchmark datasets.
Hand Gesture Recognition for Sign Language Using 3DCNN

Recently, automatic hand gesture recognition has gained increasing importance for two principal reasons: the growth of the deaf and hearing-impaired population, and the development of visionbased applications and touchless control on ubiquitous devices.As hand gesture recognition is at the core of sign language analysis a robust hand gesture recognition system should consider both spatial and temporal features.Unfortunately, finding discriminative spatiotemporal descriptors for a hand gesture sequence is not a trivial task.In this study, we proposed an efficient deep convolutional neural networks approach for hand gesture recognition.The proposed approach employed transfer learning to beat the scarcity of a large labeled hand gesture dataset.We evaluated it using three gesture datasets from color videos: 40, 23, and 10 classes were used from these datasets.The approach obtained recognition rates of 98.12%, 100%, and 76.67% on the three datasets, respectively for the signer-dependent mode.For the signer-independent mode, it obtained recognition rates of 84.38%, 34.9%, and 70% on the three datasets, respectively.
Crocodile Language Friend: Tangibles to Foster Children's Language Use

Active language use refers to people's use of a language in their everyday lives and activities. Tangible User Interfaces (TUIs) can facilitate children's participation in language activities such as language learning, communication, storytelling, and social play. However, few TUI projects take the lens of active language use, and exploit the benefits of tangibles for maintaining and revitalising endangered languages. We present the Crocodile Language Friend, co-designed with the Wujal Wujal community, to foster children's use of the Kuku Yalanji Aboriginal language. We contribute a discussion of the ways in which the crocodile's physical characteristics (e.g. size, shape, materials, and personalization) can encourage language use in individual and social activities beyond the affordances of screen-based systems.
Zero-Shot Restoration of Underexposed Images via Robust Retinex Decomposition

Underexposed images often suffer from serious quality degradation such as poor visibility and latent noise in the dark. Most previous methods for underexposed images restoration ignore the noise and amplify it during stretching contrast. We predict the noise explicitly to achieve the goal of denoising while restoring the underexposed image. Specifically, a novel three-branch convolution neural network, namely RRDNet (short for Robust Retinex Decomposition Network), is proposed to decompose the input image into three components, illumination, reflectance and noise. As an image-specific network, RRDNet doesn't need any prior image examples or prior training. Instead, the weights of RRDNet will be updated by a zero-shot scheme of iteratively minimizing a specially designed loss function. Such a loss function is devised to evaluate the current decomposition of the test image and guide noise estimation. Experiments demonstrate that RRDNet can achieve robust correction with overall naturalness and pleasing visual quality. To make the results reproducible, the source code has been made publicly available at https://aaaaangel.github.io/RRDNet-Homepage.
What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation

Unsupervised domain adaptation has attracted growing research attention on semantic segmentation. However, 1) most existing models cannot be directly applied into lesions transfer of medical images, due to the diverse appearances of same lesion among different datasets; 2) equal attention has been paid into all semantic representations instead of neglecting irrelevant knowledge, which leads to negative transfer of untransferable knowledge. To address these challenges, we develop a new unsupervised semantic transfer model including two complementary modules (i.e., T_D and T_F ) for endoscopic lesions segmentation, which can alternatively determine where and how to explore transferable domain-invariant knowledge between labeled source lesions dataset (e.g., gastroscope) and unlabeled target diseases dataset (e.g., enteroscopy). Specifically, T_D focuses on where to translate transferable visual information of medical lesions via residual transferability-aware bottleneck, while neglecting untransferable visual characterizations. Furthermore, T_F highlights how to augment transferable semantic features of various lesions and automatically ignore untransferable representations, which explores domain-invariant knowledge and in return improves the performance of T_D. To the end, theoretical analysis and extensive experiments on medical endoscopic dataset and several non-medical public datasets well demonstrate the superiority of our proposed model.
A General Knowledge Distillation Framework for Counterfactual Recommendation via Uniform Data

Recommender systems are feedback loop systems, which often face bias problems such as popularity bias, previous model bias and position bias. In this paper, we focus on solving the bias problems in a recommender system via a uniform data. Through empirical studies in online and offline settings, we observe that simple modeling with a uniform data can alleviate the bias problems and improve the performance. However, the uniform data is always few and expensive to collect in a real product. In order to use the valuable uniform data more effectively, we propose a general knowledge distillation framework for counterfactual recommendation that enables uniform data modeling through four approaches: (1) label-based distillation focuses on using the imputed labels as a carrier to provide useful de-biasing guidance; (2) feature-based distillation aims to filter out the representative causal and stable features; (3) sample-based distillation considers mutual learning and alignment of the information of the uniform and non-uniform data; and (4) model structure-based distillation constrains the training of the models from the perspective of embedded representation. We conduct extensive experiments on both public and product datasets, demonstrating that the proposed four methods achieve better performance over the baseline models in terms of AUC and NLL. Moreover, we discuss the relation between the proposed methods and the previous works. We emphasize that counterfactual modeling with uniform data is a rich research area, and list some interesting and promising research topics worthy of further exploration. Note that the source codes are available at \urlhttps://github.com/dgliu/SIGIR20_KDCRec.
GLU-Net: Global-Local Universal Network for Dense Flow and Correspondences

Establishing dense correspondences between a pair of images is an important and general problem, covering geometric matching, optical flow and semantic correspondences. While these applications share fundamental challenges, such as large displacements, pixel-accuracy, and appearance changes, they are currently addressed with specialized network architectures, designed for only one particular task. This severely limits the generalization capabilities of such networks to new scenarios, where e.g. robustness to larger displacements or higher accuracy is required. In this work, we propose a universal network architecture that is directly applicable to all the aforementioned dense correspondence problems. We achieve both high accuracy and robustness to large displacements by investigating the combined use of global and local correlation layers. We further propose an adaptive resolution strategy, allowing our network to operate on virtually any input image resolution. The proposed GLU-Net achieves state-of-the-art performance for geometric and semantic matching as well as optical flow, when using the same network and weights. Code and trained models are available at https://github.com/PruneTruong/GLU-Net.
Artificial Intelligence Image Recognition Method Based on Convolutional Neural Network Algorithm

As an algorithm with excellent performance, convolutional neural network has been widely used in the field of image processing and achieved good results by relying on its own local receptive fields, weight sharing, pooling, and sparse connections. In order to improve the convergence speed and recognition accuracy of the convolutional neural network algorithm, this paper proposes a new convolutional neural network algorithm. First, a recurrent neural network is introduced into the convolutional neural network, and the deep features of the image are learned in parallel using the convolutional neural network and the recurrent neural network. Secondly, according to the idea of ResNet's skip convolution layer, a new residual module ShortCut3-ResNet is constructed. Then, a dual optimization model is established to realize the integrated optimization of the convolution and full connection process. Finally, the effects of various parameters of the convolutional neural network on the network performance are analyzed through simulation experiments, and the optimal network parameters of the convolutional neural network are finally set. Experimental results show that the convolutional neural network algorithm proposed in this paper can learn the diverse features of the image, and improve the accuracy of feature extraction and image recognition ability of the convolutional neural network.
Explainable AI in Healthcare

Artificial Intelligence (AI) is an enabling technology that when integrated into healthcare applications and smart wearable devices such as Fitbits etc. can predict the occurrence of health conditions in users by capturing and analysing their health data. The integration of AI and smart wearable devices has a range of potential applications in the area of smart healthcare but there is a challenge in the black box operation of decisions made by AI models which have resulted in a lack of accountability and trust in the decisions made. Explainable AI (XAI) is a domain in which techniques are developed to explain predictions made by AI systems. In this paper, XAI is discussed as a technique that can used in the analysis and diagnosis of health data by AI-based systems and a proposed approach presented with the aim of achieving accountability. transparency, result tracing, and model improvement in the domain of healthcare.
The Use of Virtual Reality Alone Does Not Promote Training Performance (but Sense of Presence Does)

Virtual reality offers novel ways to develop skills and learning. This technology can be used to enhance the way we educate and train professionals by possibly being more effective, cost-efficient, and reducing training-related risks. However, the potential benefits from virtual training assume that the trained skills can be transferred to the real world. Nevertheless, in the current published scientific literature, there is limited empirical evidence that links virtual reality use to better learning. The present investigation aimed to explore the use of virtual reality as a tool for training procedural skills and compare this modality with traditional instruction methods. To investigate skill development using the two forms of training, participants were randomly divided into two groups. The first group received training through an instructional video, while the second group trained in virtual reality. After the training session, the participants performed the trained task in a real setting, and task performance was measured. Subsequently, the user’s experienced sense of presence and simulator sickness were measured with self-report questionnaires. There were no significant differences between groups for any of the performance measures. There was no gender effect on performance. Importantly, the results of the present study indicate that a high sense of presence during the virtual reality simulation might contribute to increased skill learning. These findings can be used as a starting point that could be of value when further exploring virtual reality as a tool for skill development.
MFS-MCDM: Multi-label feature selection using multi-criteria decision making

In this paper, for the first time, a feature selection procedure is modeled as a multi-criteria decision making (MCDM) process. This method is applied to a multi-label data and we have used the TOPSIS (Technique of Order Preference by Similarity to Ideal Solution) method as a famous MCDM algorithm to evaluate the features based on their relationship with multiple labels as different criteria. Our proposed method, Multi-label Feature Selection using Multi-Criteria Decision Making (MFS-MCDM) which treated the multi-label feature selection as an information fusion process, first obtains a decision matrix using the ridge regression algorithm and then calculates the weight of each column of this matrix based on the entropy of each label. Then, the TOPSIS approach is used to assign a score to each feature based on the weighted decision matrix. Finally, a rank vector for the features is generated as output which the user can select a desired number of features. The superiority of the proposed method is shown in experimental results comparing to other similar methods in terms of all evaluation metrics.
Virtual reality approaches to pain: toward a state of the science

aVirginia Commonwealth University, Richmond, VA, United States bOhio University, Athens, OH, United States cUniversity of Alabama at Birmingham, Birmingham, AL, United States dImmersive Experience Labs, LLC, Birmingham, AL, United States *Corresponding author. Address: Department of Physical Medicine and Rehabilitation, Box 980677, VCU School of Medicine, Richmond, VA 23298-0677, United States. Tel.: 917-209-4866. E-mail address: [email protected] (Z. Trost). Sponsorships or competing interests that may be relevant to content are disclosed at the end of this article. Supplemental digital content is available for this article. Direct URL citations appear in the printed text and are provided in the HTML and PDF versions of this article on the journal's Web site (www.painjournalonline.com).