Reviewerly/exHarmony huggingface dataset:
Air-Writing Recognition Based on Deep Convolutional Neural Networks

Air-writing recognition has received wide attention due to its potential application in intelligent systems. To date, some of the fundamental problems in isolated writing have not been addressed effectively. This paper presents a simple yet effective air-writing recognition approach based on deep convolutional neural networks (CNNs). A robust and efficient hand tracking algorithm is proposed to extract air-writing trajectories collected by a single web camera. The algorithm addresses the push-to-write problem and avoids restrictions on the users’ writing without using a delimiter and an imaginary box. A novel preprocessing scheme is also presented to convert the writing trajectory into appropriate forms of data, making the CNNs trained with these forms of data simpler and more effective. Experimental results indicate that the proposed approach not only obtains much higher recognition accuracy but also reduces the network complexity significantly compared to the popular image-based methods.
A region-based convolutional network for nuclei detection and segmentation in microscopy images

Automated detection and segmentation of nuclei in microscopy images are of significant importance to biomedical research and clinical practice, including nuclear morphology analysis, cancer diagnosis and grading. However, these tasks are still challenging due to large numbers of adhered and clustered nuclei. Modern CNN-based nuclei detection and segmentation methods rely on bounding box regression and non-maximum suppression to locate the nuclei, which would lead to inferior localized bounding boxes of the adhered and clustered nuclei. In this paper, we propose a region-based convolutional network to tackle this challenge. In particular, a GA-RPN module integrating the guided anchoring(GA) into the region proposal network(RPN) is employed to generate candidate proposals that are more suitable for nuclei detection. A new branch is proposed to regress the intersection over union(IoU) between the detection boxes and their corresponding ground truth for locating the bounding box. To reduce the undetection of adhered and clustered nuclei, we pass a fusioned box score(FBS) into soft non-maximum suppression(SoftNMS) to preserve the true positive candidate boxes. The experiments are performed on two public challenging datasets, which are designed to challenge an algorithm's ability to generalize across different varieties. The results empirically demonstrate that our method has better detection and segmentation capability than existing state-of-the-art methods. In conclusion, our method can improve the performances and provide a potential for sophisticated cell-level analysis within digital tissue images.
Exploring virtual reality for quality immersive empathy building experiences

Virtual reality (VR) technology presents users with virtual environments to experience various interactive, immersive, and imaginary experiences. While traditional perspective-taking exercises rely on the participant to imagine a self-other merging process to feel connected with other people (typically using second and third-person narrative perspectives), VR can allow an individual to embody an other through first-person narratives delivered via multimodal – visual, aural, haptic – technology-mediated experiences. This process enables users to perceptually and effectively portal into somebody else's body, where they can potentially see, hear, and feel from the point of view of the protagonist and control choices on their behalf in real-time. This article explores the use of VR as an ‘empathy-making machine’ by facilitating perspective-taking and allowing users to experience another person's circumstances. An experiment was performed to compare two different types of perspective-taking VR applications. Levels of empathy, oneness, and attitudes towards a protagonist or focus group within VR materials were captured. Participants then identified the elements of the VR content that contributed to a quality experience. These measures were used to discuss methodologies and techniques for creating quality empathy-building techniques. The findings of this research will be used to inform future creative technology projects presented in VR.
Image sentiment classification via multi-level sentiment region correlation analysis

Human’s understanding of image content is a multi-level and multi-stage process. For visual sentiment analysis, this process can be specified as the gradual perception from semantic to emotion of regions in an image. The mining of emotion-related regions is valuable for sentiment recognition, and it is even more important to further investigate the semantic associations formed between these regions. In this paper, we propose a novel multi-level sentiment region correlation analysis model, which exploits the regions in an image that are most potentially affected by emotions from multiple perspectives and motivates the interaction between sentiment regions. It makes the visual content of multi-level sentiment regions and the implicit correlations within them robust cues for image sentiment recognition. We innovatively propose a module of correlation analysis of multi-level sentiment regions to exploit the effects of higher-order and rich interactions on emotions with encoders of the Transformer. Experiments on a variety of public visual sentiment analysis datasets at different scales show that the proposed MSRCA model achieve excellent performance in image sentiment classification and outperforms other existing methods.
Kernelized Supervised Laplacian Eigenmap for Visualization and Classification of Multi-Label Data

We had previously proposed a supervised Laplacian eigenmap for visualization (SLE-ML) that can handle multi-label data. In addition, SLE-ML can control the trade-off between the class separability and local structure by a single trade-off parameter. However, SLE-ML cannot transform new data, that is, it has the "out-of-sample" problem. In this paper, we show that this problem is solvable, that is, it is possible to simulate the same transformation perfectly using a set of linear sums of reproducing kernels (KSLE-ML) with a nonsingular Gram matrix. We experimentally showed that the difference between training and testing is not large; thus, a high separability of classes in a low-dimensional space is realizable with KSLE-ML by assigning an appropriate value to the trade-off parameter. This offers the possibility of separability-guided feature extraction for classification. In addition, to optimize the performance of KSLE-ML, we conducted both kernel selection and parameter selection. As a result, it is shown that parameter selection is more important than kernel selection. We experimentally demonstrated the advantage of using KSLE-ML for visualization and for feature extraction compared with a few typical algorithms.
GitHub Discussions: An exploratory study of early adoption

Abstract Discussions is a new feature of GitHub for asking questions or discussing topics outside of specific Issues or Pull Requests. Before being available to all projects in December 2020, it had been tested on selected open source software projects. To understand how developers use this novel feature, how they perceive it, and how it impacts the development processes, we conducted a mixed-methods study based on early adopters of GitHub discussions from January until July 2020. We found that: (1) errors, unexpected behavior, and code reviews are prevalent discussion categories; (2) there is a positive relationship between project member involvement and discussion frequency; (3) developers consider GitHub Discussions useful but face the problem of topic duplication between Discussions and Issues; (4) Discussions play a crucial role in advancing the development of projects; and (5) positive sentiment in Discussions is more frequent than in Stack Overflow posts. Our findings are a first step towards data-informed guidance for using GitHub Discussions, opening up avenues for future work on this novel communication channel.
DACHA: A Dual Graph Convolution Based Temporal Knowledge Graph Representation Learning Method Using Historical Relation

Temporal knowledge graph (TKG) representation learning embeds relations and entities into a continuous low-dimensional vector space by incorporating temporal information. Latest studies mainly aim at learning entity representations by modeling entity interactions from the neighbor structure of the graph. However, the interactions of relations from the neighbor structure of the graph are neglected, which are also of significance for learning informative representations. In addition, there still lacks an effective historical relation encoder to model the multi-range temporal dependencies. In this article, we propose a d ual gr a ph c onvolution network based TKG representation learning method using h istorical rel a tions (DACHA). Specifically, we first construct the primal graph according to historical relations, as well as the edge graph by regarding historical relations as nodes. Then, we employ the dual graph convolution network to capture the interactions of both entities and historical relations from the neighbor structure of the graph. In addition, the temporal self-attentive historical relation encoder is proposed to explicitly model both local and global temporal dependencies. Extensive experiments on two event based TKG datasets demonstrate that DACHA achieves the state-of-the-art results.
Ensemble Gaussian Processes for Online Learning Over Graphs With Adaptivity and Scalability

In the past decade, semi-supervised learning (SSL) over graphs has gained popularity due to its importance in a gamut of network science applications. While most of existing SSL methods provide only point estimates of the targeted variables, the present work capitalizes on Gaussian processes (GPs) to offer a Bayesian SSL approach over graphs with uncertainty quantification, a key attribute especially in safety-critical domains. To accommodate also delay-sensitive scenarios, an incremental learning mode is considered, where prediction of the desired value of a new node per iteration is followed by processing the corresponding nodal observation. Taking the per-node one-hop connectivity vector as the input, the prediction of targeted nodal value is enabled by leveraging an ensemble (E) of GP experts, whose weights are updated in a data-adaptive fashion. In the resultant GRaph-ADpative EGP framework, random feature-based kernel approximation is employed to not only allow learning with scalability, but also preserve privacy by relying on an encrypted version of each node’s connectivity. Besides the one-hop connectivity vector, the novel GradEGP accommodates each node’s egonet features as alternative inputs. On the analytical side, to assess the performance of GradEGP in the adversarial setting where the generative assumptions are violated, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">regret analysis</i> measures the cumulative online losses relative to their counterparts of a benchmark learner with batch data in hindsight. Tests conducted on real and synthetic datasets demonstrate the effectiveness of the advocated method.
CMML

Practical recommender systems experience a cold-start problem when observed user-item interactions in the history are insufficient. Meta learning, especially gradient based one, can be adopted to tackle this problem by learning initial parameters of the model and thus allowing fast adaptation to a specific task from limited data examples. Though with significant performance improvement, it commonly suffers from two critical issues: the non-compatibility with mainstream industrial deployment and the heavy computational burdens, both due to the inner-loop gradient operation. These two issues make them hard to be applied in practical recommender systems. To enjoy the benefits of meta learning framework and mitigate these problems, we propose a recommendation framework called Contextual Modulation Meta Learning (CMML). CMML is composed of fully feed-forward operations so it is computationally efficient and completely compatible with the mainstream industrial deployment. CMML consists of three components, including a context encoder that can generate context embedding to represent a specific task, a hybrid context generator that aggregates specific user-item features with task-level context, and a contextual modulation network, which can modulate the recommendation model to adapt effectively. We validate our approach on both scenario-specific and user-specific cold-start setting on various real-world datasets, showing CMML can achieve comparable or even better performance with gradient based methods yet with higher computational efficiency and better interpretability.
First Assessment Results of Surveying Engineering Labs in Immersive and Interactive Virtual Reality

Technological advancements in virtual reality have opened the door for widespread dissemination and application in engineering education. In surveying engineering, outdoor labs are essential, because they offer great means for experiential learning and preparation for real applications. However, challenges related to weather, inaccessibility to sites, transportation costs, and liability reduce time spent with instruments, and limit comprehension of surveying procedures and preparedness for the job market. Through virtual reality, we can create realistic surveying scenarios that are difficult to implement in practice, such as surveying in cities, at construction sites, and in different terrain scenarios. In addition, virtual reality can be used to support remote and online learning. This paper presents the thorough implementation of virtual reality labs in a surveying engineering first-year class. Technical feedback showed that surveying labs were recreated with a high level of fidelity, which is important to transfer skills learned in virtual reality to the real environment. Students stated that virtual reality gave them more incentives to learn, added to the fun of learning, and improved their overall learning experience. Furthermore, students indicated that virtual reality labs helped them understand surveying procedures and the operation of instruments, and they considered virtual labs to be a useful tool for preparation for physical labs. Pedagogical assessments using grades from previous years showed that the virtual reality labs enhanced student learning. Barriers of virtual reality, such as nausea and motion sickness, limit their extensive use. Despite a few negatives, virtual reality offers viable solutions to challenges in surveying engineering education, with the potential for further development and integration in other surveying courses.
Identifying and addressing digital health risks associated with emergency pandemic response: Problem identification, scoping review, and directions toward evidence-based evaluation

The COVID-19 pandemic has accelerated digital health applications in multifaceted disease management dimensions. This study aims (1) to identify risk issues relating to the rapid development and redeployment of COVID-19 related e-health systems, in primary care, and in the health ecosystems interacting with it and (2) to suggest evidence-based evaluation directions under emergency response.After initial brainstorming of digital health risks posed in this pandemic, a scoping review method was adopted to collect evidence across databases of PubMed, CINAHL, and EMBASE. Peer-review publications, reports, news sources, and websites that credibly identified the challenges relating digital health scaled for COVID-19 were scrutinized. Additional supporting materials were obtained through snowball sampling and the authors' global digital health networks. Studies satisfying the selection criteria were charted based on their study design, primary care focus, and coverage of e-health areas of risk.Fifty-eight studies were mapped for qualitative synthesis. Five identified digital health risk areas associated with the pandemic were governance, system design and coordination, information access, service provision, and user (professional and public) reception. We observed that rapid digital health responses may embed challenges in health system thinking, the long-term development of digital health ecosystems, and interoperability of health IT infrastructure, with concomitant weaknesses in existing evaluation theories.Through identifying digital health risks posed during the pandemic, this paper discussed potential directions for next-generation informatics evaluation development, to better prepare for the post-COVID-19 era, a new future epidemic, or other unforeseen global health emergencies. An updated evidence-based approach to health informatics is essential to gain public confidence in digital health across primary and other health sectors.
CrossLine: Breaking "Security-by-Crash" based Memory Isolation in AMD SEV

AMD's Secure Encrypted Virtualization (SEV) is an emerging security feature of modern AMD processors that allows virtual machines to run with encrypted memory and perform confidential computing even with an untrusted hypervisor. This paper first demystifies SEV's improper use of address space identifier (ASID) for controlling accesses of a VM to encrypted memory pages, cache lines, and TLB entries. We then present the CROSSLINE attacks, a novel class of attacks against SEV that allow the adversary to launch an attacker VM and change its ASID to that of the victim VM to impersonate the victim. We present two variants of CROSSLINE attacks: CROSSLINE V1 decrypts victim's page tables or any memory blocks conforming to the format of a page table entry; CROSSLINE V2 constructs encryption and decryption oracles by executing instructions of the victim VM. We discuss the applicability of CROSSLINE attacks on AMD's SEV, SEV-ES, and SEV-SNP processors.
Smart Director: An Event-Driven Directing System for Live Broadcasting

Live video broadcasting normally requires a multitude of skills and expertise with domain knowledge to enable multi-camera productions. As the number of cameras keeps increasing, directing a live sports broadcast has now become more complicated and challenging than ever before. The broadcast directors need to be much more concentrated, responsive, and knowledgeable, during the production. To relieve the directors from their intensive efforts, we develop an innovative automated sports broadcast directing system, called Smart Director, which aims at mimicking the typical human-in-the-loop broadcasting process to automatically create near-professional broadcasting programs in real-time by using a set of advanced multi-view video analysis algorithms. Inspired by the so-called “three-event” construction of sports broadcast [ 14 ], we build our system with an event-driven pipeline consisting of three consecutive novel components: (1) the Multi-View Event Localization to detect events by modeling multi-view correlations, (2) the Multi-View Highlight Detection to rank camera views by the visual importance for view selection, and (3) the Auto-Broadcasting Scheduler to control the production of broadcasting videos. To our best knowledge, our system is the first end-to-end automated directing system for multi-camera sports broadcasting, completely driven by the semantic understanding of sports events. It is also the first system to solve the novel problem of multi-view joint event detection by cross-view relation modeling. We conduct both objective and subjective evaluations on a real-world multi-camera soccer dataset, which demonstrate the quality of our auto-generated videos is comparable to that of the human-directed videos. Thanks to its faster response, our system is able to capture more fast-passing and short-duration events which are usually missed by human directors.
Meteor: Cryptographically Secure Steganography for Realistic Distributions

Despite a long history of research and wide-spread applications to censorship resistant systems, practical steganographic systems capable of embedding messages into realistic communication distributions, like text, do not exist. We identify two primary impediments to deploying universal steganography: (1) prior work leaves the difficult problem of finding samplers for non-trivial distributions unaddressed, and (2) prior constructions have impractical minimum entropy requirements. We investigate using generative models as steganographic samplers, as they represent the best known technique for approximating human communication. Additionally, we study methods to overcome the entropy requirement, including evaluating existing techniques and designing a new steganographic protocol, called Meteor. The resulting protocols are provably indistinguishable from honest model output and represent an important step towards practical steganographic communication for mundane communication channels. We implement Meteor and evaluate it on multiple computation environments with multiple generative models.
Z3str4: A Multi-armed String Solver

We present Z3str4, a new high-performance string SMT solver for a rich quantifier-free first-order theory of strings and length constraints. These kinds of constraints have found widespread application in analysis of string-intensive programs in general, and web applications in particular. Three key contributions underpin our solver: first, a novel length-abstraction algorithm that performs various string-length based abstractions and refinements along with a bit-vector backend; second, an arrangement-based solver with a bit-vector backend; third, an algorithm selection and constraint-sharing architecture which leverages the above-mentioned solvers along with the Z3 sequence (Z3seq) solver. We perform extensive empirical evaluation over 20 different industrial and randomly-generated benchmarks with over 120,000+ instances, and show that Z3str4 outperforms the previous best solvers, namely, CVC4, Z3seq, and Z3str3 in both total solved instances and total runtime.
Driver Gaze Zone Estimation via Head Pose Fusion Assisted Supervision and Eye Region Weighted Encoding

Driver gaze zone estimation is an important task in Advanced Driver Assistance Systems (ADAS), which suffers difficulties including head pose, capture direction, glass occlusion, and real-time requirement, etc. Most previous methods combine face modalities and head pose using concat process, which may result in over-fitting due to the unbalanced dimension. Focusing on gaze zone estimation problems, we propose the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Head Pose Fusion Assisted supervision &amp; Eye Region Weighted Encoding</i> ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HP-ERW</b> ) structure, which fuses head pose attribute and face modalities together through spatial attention and Kronecker product mechanisms. Firstly, we introduce a pre-processing module dealing with head pose and face information, with the purpose of extracting input vectors and improving the fusion speed of the HP-ERW structure. Secondly, an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Eye Region Weighted Encoding Network</i> (ERW-Net) based on spatial attention is proposed to strengthen the networks perception ability for encoding features. Finally, we propose a dual-channel <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Head Pose Fusion Network</i> ( <bold xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">HP-Net</b> ) based on the Kronecker product mechanism, with the purpose of fusing head pose and improving the estimation accuracy. Experiments show that the HP-ERW outperforms compared existing methods on several public datasets. The designed ADAS using the proposed method achieves 23.5 fps real-time application with small memory requirement of 4,884 KB.