Reviewerly/exHarmony huggingface dataset:
Deep Object Detection with Example Attribute Based Prediction Modulation

Deep object detectors suffer from the gradient contribution imbalance during training. In this paper, we point out that such imbalance can be ascribed to the imbalance in example attributes, e.g., difficulty and shape variation degree. We further propose example attribute based prediction modulation (EAPM) to address it. In EAPM, first, the attribute of an example is defined by the prediction and the corresponding ground truth. Then, a modulating factor w.r.t the example attribute is introduced to modulate the prediction error. Finally, the new prediction and the ground-truth are input into the loss function. Essentially, we adjust the gradients of examples with specific attributes to reweight their contribution on the global gradients. We apply EAPM with focal loss and balanced L1 loss to simultaneously solve the imbalance in classification and localization. The experimental results on MS COCO demonstrate that EAPM can bring substantial improvement for deep object detectors.
Trading behavior in bitcoin futures: Following the “smart money”

Abstract The Bitcoin futures market has grown rapidly since its 2017 introduction. Along with enabling institutional traders to access a regulated cryptocurrency product, futures provide a means to improve market efficiency by shorting Bitcoin. We examine trading behavior in Bitcoin futures utilizing the Commodity Futures Trading Commission Commitment of Traders report. Leveraged money traders tend to hold the largest positions, be net short, and their trading behavior plays a key role in the Bitcoin futures market. Our empirical results show that leveraged money traders display market timing ability, largely by adjusting their short positions. It seems that other trader types follow this “smart money” in adjusting their own positions in subsequent periods. We also demonstrate that it is possible to construct profitable trading strategies based on observed variations in leveraged money positions.
Adversarial Mask Transformer for Sequential Learning

Mask language model has been successfully developed to build a transformer for robust language understanding. The transformer-based language model has achieved excellent results in various downstream applications. However, typical mask language model is trained by predicting the randomly masked words and is used to transfer the knowledge from rich-resource pre-training task to low-resource downstream tasks. This study incorporates a rich contextual embedding from pre-trained model and strengthens the attention layers for sequence-to-sequence learning. In particular, an adversarial mask mechanism is presented to deal with the shortcoming of random mask and accordingly enhance the robustness in word prediction for language understanding. The adversarial mask language model is trained in accordance with a minimax optimization over the word prediction loss. The worst-case mask is estimated to build an optimal and robust language model. The experiments on two machine translation tasks show the merits of the adversarial mask transformer.
EMOQ-TTS: Emotion Intensity Quantization for Fine-Grained Controllable Emotional Text-to-Speech

Although recent advances in text-to-speech (TTS) have shown significant improvement, it is still limited to emotional speech synthesis. To produce emotional speech, most works utilize emotion information extracted from emotion labels or reference audio. However, they result in monotonous emotional expression due to the utterance-level emotion conditions. In this paper, we propose EmoQ-TTS, which synthesizes expressive emotional speech by conditioning phoneme-wise emotion information with fine-grained emotion intensity. Here, the intensity of emotion information is rendered by distance-based intensity quantization without human labeling. We can also control the emotional expression of synthesized speech by conditioning intensity labels manually. The experimental results demonstrate the superiority of EmoQ-TTS in emotional expressiveness and controllability.
Improving Self-Supervised Learning for Speech Recognition with Intermediate Layer Supervision

Recently, pioneer work finds that self-supervised pre-training methods can improve multiple downstream speech tasks, because the model utilizes bottom layers to learn speaker-related information and top layers to encode content-related information. Since the network capacity is limited, we believe the speech recognition performance could be further improved if the model is dedicated to audio content information learning. To this end, we propose Intermediate Layer Supervision for Self-Supervised Learning (ILS-SSL), which forces the model to concentrate on content information as much as possible by adding an additional SSL loss on the intermediate layers. Experiments on LibriSpeech test-other set show that our method outperforms HuBERT significantly, which achieves a 23.5%/11.6% relative word error rate reduction in the w/o language model setting for Base/Large models. Detailed analysis shows the bottom layers of our model have a better correlation with phonetic units, which is consistent with our intuition and explains the success of our method for ASR. We will release our code and model at https://github.com/microsoft/UniSpeech.
Efficient 3D human pose estimation from RGBD sensors

Human pose estimation is a core component in applications for which some level of human–computer interaction is required, such as assistive robotics, ambient assisted living or the motion capture systems used in biomechanics or video games production. In this paper, we propose an end-to-end pipeline for estimating 3D human poses that works in real-time in an off-the-shelf computer, using as input video sequences captured with a commercial RGBD sensor. Our hybrid approach is composed of two stages: 2D pose estimation using deep neural networks and 3D registration, for which a lightweight algorithm based on classic computer vision techniques has been developed. We compare several 2D pose estimators and validate the performance of our proposed method against the state-of-the-art, using as benchmark an international and publicly available dataset. Our 2D to 3D registration module alone can reach frame rates of up to 99 fps, while achieving an average error per joint of 132 mm. Furthermore, the proposed solution is agnostic to the model used for 2D pose estimation and can be upgraded with new upcoming solutions or adapted for different articulated objects.
Post-symptomatic detection of COVID-2019 grade based mediative fuzzy projection

The concept of fuzzy set, intuitionistic set, and mediative fuzzy set as a generalization of a crisp set have been introduced in many real-life applications. The concept of crisp relation between elements of sets can be extended to fuzzy relations. Extended relations will be considered as relations on fuzzy sets. In this work, we developed the concept of mediative fuzzy relation and meditative fuzzy projection in the context of fuzzy relation and fuzzy projection. We extended the basic operations of fuzzy projection into intuitionistic fuzzy projection and then in the mediative fuzzy projection. We have shown the credibility and impact of mediative index factor involves in the mediative fuzzy projection in context of prediction work in relation to the proposed model. Further, we applied the mediative fuzzy projection in the medical diagnosis in post-COVID-19 patients. The obtained results have also been discussed with their geometrical representation.
Expert Systems and Neural Networks and their Impact on the Relevance of Financial Information in the Jordanian Commercial Banks

The current study aims to discern the impact of expert systems and neural network on the Jordanian commercial banks. In achieving the objective, the study employed descriptive analytical approach and the population consisted of the 13 Jordanian commercial banks listed at Amman Stock Exchange-ASE. The primary data were obtained by using a questionnaire with 188 samples distributed to a group of accountants, internal auditors, and programmers, who constitute the study sample. The results unveiled that there is an impact of the application of expert systems and neural networks on the relevance of financial information in Jordanian commercial banks. It also revealed that there is a high level of relevance of financial information in Jordanian commercial banks. Accordingly, the study recommended the need for banks to keep pace with the progress and development taking place in connection to the process and environment of expertise systems by providing modern and developed devices to run various programs and expert systems. It also recommended that, Jordanian commercial banks need to rely more on advanced systems to operate neural network technology more efficiently.
Serious Games as a Method for Enhancing Learning Engagement: Student Perception on Online Higher Education During COVID-19

The COVID-19 pandemic has enforced social isolation in many countries worldwide, which forced teachers at all levels of education, including the university context, to adapt new teaching strategies. This study presents a method developed in this regard, that is, serious games were used as a complement to synchronous online classes to ensure the continuity of pedagogical activities in a physiology course at Universidad Andrés Bello, Chile. Using serious games is a strategy in the field of gamification, which is a commonly used learning strategy for online teaching as necessitated by COVID-19. This study is quantitative in nature and conducted a questionnaire survey on 108 second-year undergraduate nursing students to determine their perception about this innovation. The results demonstrate that the students well valued the proposed pedagogical innovative model in terms of motivation and engagement. Moreover, they reported that the model can serve as a meaningful learning experience. These perceptions suggest that the model is an efficient strategy for implementing the physiology curricula in the context of online teaching. Moreover, the results imply that the model should be applied to other courses and disciplines in the undergraduate program and provide support that it is a valid strategy for face-to-face teaching. Lastly, the finding points to the potential of the model to be explored as a learning strategy in the age of education post-COVID-19.
DeepCore: A Comprehensive Library for Coreset Selection in Deep Learning

Coreset selection, which aims to select a subset of the most informative training samples, is a long-standing learning problem that can benefit many downstream tasks such as data-efficient learning, continual learning, neural architecture search, active learning, etc. However, many existing coreset selection methods are not designed for deep learning, which may have high complexity and poor generalization performance. In addition, the recently proposed methods are evaluated on models, datasets, and settings of different complexities. To advance the research of coreset selection in deep learning, we contribute a comprehensive code library (The code is available in https://github.com/PatrickZH/DeepCore .), namely DeepCore, and provide an empirical study on popular coreset selection methods on CIFAR10 and ImageNet datasets. Extensive experiments on CIFAR10 and ImageNet datasets verify that, although various methods have advantages in certain experiment settings, random selection is still a strong baseline.
Use Cases for Design Personas: A Systematic Review and New Frontiers

Personas represent the needs of users in diverse populations and impact design by endearing empathy and improving communication. While personas have been lauded for their benefits, we could locate no prior review of persona use cases in design, prompting the question: how are personas actually used to achieve these benefits? To address this question, we review 95 articles containing persona application across multiple domains, and identify software development, healthcare, and higher education as the top domains that employ personas. We then present a three-stage design hierarchy of persona usage to describe how personas are used in design tasks. Finally, we assess the increasing trend of persona initiatives aimed towards social good rather than solely commercial interests. Our findings establish a roadmap of best practices for how practitioners can innovatively employ personas to increase the value of designs and highlight avenues of using personas for socially impactful purposes.
Exploring the relationship between children's facial emotion processing characteristics and speech communication ability using deep learning on eye tracking and speech performance measures

The ability of efficient facial emotion recognition (FER) plays a significant role in successful human communication and is closely associated with multiple speech communication disorders (SCD) in children. Despite the relevance, little is known about how speech communication abilities (SCA) and FER are correlated or of their underlying mechanism. To address this, we monitored eye movements of 115 children while watching human faces with different emotions and designed a machine-learning based SCD prediction model to explore the underlying pattern of eye movements during the FER task as well as their correlation with SCA. Strong and detailed correlations were found between different dimensions of SCA and various eye-movement features. A group of FER gazing patterns was found to be highly sensitive to the possibility of children's SCD. The SCD prediction model reached an accuracy as high as 88.9%, which offers a possible technique to fast screen SCD for children.
There Is No First- or Third-Person View in Virtual Reality: Understanding the Perspective Continuum

Modern games make creative use of First- and Third-person perspectives (FPP and TPP) to allow the player to explore virtual worlds. Traditionally, FPP and TPP perspectives are seen as distinct concepts. Yet, Virtual Reality (VR) allows for flexibility in choosing perspectives. We introduce the notion of a perspective continuum in VR, which is technically related to the camera position and conceptually to how users perceive their environment in VR. A perspective continuum enables adapting and manipulating the sense of agency and involvement in the virtual world. This flexibility of perspectives broadens the design space of VR experiences through deliberately manipulating perception. In a study, we explore users' attitudes, experiences and perceptions while controlling a virtual character from the two known perspectives. Statistical analysis of the empirical results shows the existence of a perspective continuum in VR. Our findings can be used to design experiences based on shifts of perception.
Great Chain of Agents: The Role of Metaphorical Representation of Agents in Conversational Crowdsourcing

Conversational agents are being widely adopted across several domains to serve a variety of purposes ranging from providing intelligent assistance to companionship. Recent literature has shown that users develop intuitive folk theories and a metaphorical understanding of conversational agents (CAs) due to the lack of a mental model of the agents. However, investigation of metaphorical agent representation in the HCI community has mainly focused on the human level, despite non-human metaphors for agents being prevalent in the real world. We adopted Lakoff and Turner’s ‘Great Chain of Being’ framework to systematically investigate the impact of using non-human metaphors to represent conversational agents on worker engagement in crowdsourcing marketplaces. We designed a text-based conversational agent that assists crowd workers in task execution. Through a between-subjects experimental study (N = 341), we explored how different human and non-human metaphors affect worker engagement, the perceived cognitive load of workers, intrinsic motivation, and their trust in the agents. Our findings bridge the gap of how users experience CAs with non-human metaphors in the context of conversational crowdsourcing.
Math Augmentation: How Authors Enhance the Readability of Formulas using Novel Visual Design Practices

With the increasing growth and impact of machine learning and other math-intensive fields, it is more important than ever to broaden access to mathematical notation. Can new visual and interactive displays help a wider readership successfully engage with notation? This paper provides the first detailed qualitative analysis of math augmentation—the practice of embellishing notation with novel visual design patterns to improve its readability. We present two qualitative studies of the practice of math augmentation. First is an analysis of 1.1k augmentations to 281 formulas in 47 blogs, textbooks, and other documents containing mathematical expressions. Second is an interview study with 12 authors who had previously designed custom math augmentations ("maugs"). This paper contributes a comprehensive inventory of the kinds of maugs that appear in math documents, and a detailed account of how authors' tools ought to be redesigned to support efficient creation of math augmentations. These studies open a critical new design space for HCI researchers and interface designers.
A hybrid consensus algorithm for master–slave blockchain in a multidomain conversation system

Conversation systems play an important role in the research and development of multidomain systems, in which the number of conversations increases rapidly with the number of domains. Multiple experts must maintain these conversation interactions, otherwise experts from different domains are unable to interact properly. For this reason, ensuring secure and efficient cooperation between different experts has become a serious issue that needs to be resolved. This study proposes a hybrid consensus algorithm for a master–slave chain to improve the efficiency and scalability of multidomain conversation interactions. Firstly, we build a master–slave chain model for a multidomain conversation system which can be applied to process multiple conversation interactions from different domains concurrently. Subsequently, a new hybrid consensus algorithm is proposed which combines reputation-driven voting, dynamic construction strategies and an incentive scheme with both economic and non-economic rewards. Implemented and tested on a local private blockchain network, the experimental results demonstrate the feasibility and effectiveness of the proposed approach, in order to offer a decentralised, scalable, multidomain conversation system in a secure manner.
A Large-Scale Measurement of Cybercrime Against Individuals

We know surprisingly little about the prevalence and severity of cybercrime in the U.S. Yet, in order to prioritize the development and distribution of advice and technology to protect end users, we require empirical evidence regarding cybercrime. Measuring crime, including cybercrime, is a challenging problem that relies on a combination of direct crime reports to the government – which have known issues of under-reporting – and assessment via carefully-designed self-report surveys. We report on the first large-scale, nationally representative academic survey (n=11,953) of consumer cybercrime experiences in the U.S. Our analysis answers four research questions: (1) What is the prevalence and (2) the monetary impact of these cybercrimes we measure in the U.S.?, (3) Do inequities exist in victimization?, and (4) Can we improve cybercrime measurement by leveraging social-reporting techniques used to measure physical crime? Our analysis also offers insight toward improving future measurement of cybercrime and protecting users.
Time-Triggered Federated Learning Over Wireless Networks

The newly emerging federated learning (FL) framework offers a new way to train machine learning models in a privacy-preserving manner. However, traditional FL algorithms are based on an event-triggered aggregation, which suffers from stragglers and communication overhead issues. To address these issues, in this paper, we present a time-triggered FL algorithm (TT-Fed) over wireless networks, which is a generalized form of classic synchronous and asynchronous FL. Taking the constrained resource and unreliable nature of wireless communication into account, we jointly study the user selection and bandwidth optimization problem to minimize the FL training loss. To solve this joint optimization problem, we provide a thorough convergence analysis for TT-Fed. Based on the obtained analytical convergence upper bound, the optimization problem is decomposed into tractable sub-problems with respect to each global aggregation round, and finally solved by our proposed online search algorithm. Simulation results show that compared to asynchronous FL (FedAsync) and FL with asynchronous user tiers (FedAT) benchmarks, our proposed TT-Fed algorithm improves the converged test accuracy by up to 12.5% and 5%, respectively, under highly imbalanced and non-IID data, while substantially reducing the communication overhead.
On the identification of accessibility bug reports in open source systems

Today, mobile devices provide support to disabled people to make their life easier due to their high accessibility and capability, e.g., finding accessible locations, picture and voice-based communication, customized user interfaces and vocabulary levels. These accessibility frameworks are directly integrated, as libraries, in various apps, providing them with accessibility functions. Just like any other software, these frameworks regularly encounter errors. These errors are reported by app developers in the form of bug reports. These bug reports related to accessibility faults need to be urgently fixed since their existence significantly hinders the usability of apps. In this context, the manual inspection of a large number of bug reports to identify accessibility-related ones is time-consuming and error-prone. Prior research has investigated mobile app user reviews classification for various purposes, including bug reports identification, feature request identification, app performance optimization etc. Yet, none of the prior research has investigated the identification of accessibility-related bug reports, making their prioritization and timely correction difficult for software developers. To support developers with this manual process, the goal of this paper is to automatically detect, for a given bug report, whether it is about accessibility or not. Thus, we tackle the identification of accessibility bug reports as a binary classification problem. To build our model, we rely on an existing dataset of manually curated accessibility bug reports, extracted from popular open-source projects, namely Mozilla Firefox and Google Chromium. We design our solution to learn from these reports the appropriate discriminative features i.e., keywords that properly represent accessibility issues. Our trained model is evaluating using stratified cross-validation, and the findings show that our classifier achieves high F1-scores of 93%.
CatchLive: Real-time Summarization of Live Streams with Stream Content and Interaction Data

Live streams usually last several hours with many viewers joining in the middle. Viewers who join in the middle often want to understand what has happened in the stream. However, catching up with the earlier parts is challenging because it is difficult to know which parts are important in the long, unedited stream while also keeping up with the ongoing stream. We present CatchLive, a system that provides a real-time summary of ongoing live streams by utilizing both the stream content and user interaction data. CatchLive provides viewers with an overview of the stream along with summaries of highlight moments with multiple levels of detail in a readable format. Results from deployments of three streams with 67 viewers show that CatchLive helps viewers grasp the overview of the stream, identify important moments, and stay engaged. Our findings provide insights into designing summarizations of live streams reflecting their characteristics.