Reviewerly/exHarmony huggingface dataset:
A Novel Micro-Expression Recognition Approach Using Attention-Based Magnification-Adaptive Networks

Micro-Expression recognition (MER) is a challenging task due to the short duration and low intensity of Micro-Expressions. A popular method to tackle this is magnifying MEs so as to enlarge the expression intensity to make recognition easier. However, the single fixed magnification strategy, widely used in existing works of MER, is not appropriate for different subjects, because each subject has specific expression intensity corresponding to different MEs. To cope with this issue, we propose a novel Attention-based Magnification-Adaptive Network (AMAN) to learn adaptive magnification levels for the ME representation. The network consists of two modules: magnification attention (MA module) to adaptively focus on appropriate magnification levels of different MEs, and frame attention (FA module) to focus on discriminative aggregated frames in a ME video. Extensive experiments on three widely used databases manifest that our method yields state-of-art results compared with other methods.
Reflection in Theory and Reflection in Practice: An Exploration of the Gaps in Reflection Support among Personal Informatics Apps

Personal informatics (PI) systems have been developed to support reflection. While reflection is considered an indispensable activity in PI use, how and when reflection occurs is still under-studied. In this paper, we present an analysis of the interactive features of 123 commercial PI apps, revealing that reflective practices are unevenly supported. The lack of features that encourage user-driven reflection, scaffolding for setting goals and configuring data collection and presentation, and consideration of wider implications stand to limit meaning-making and frustrate nuanced insight generation based on lived experiences. Based on our findings, we discuss how reflection is currently misrepresented in personal informatics tools, identify and characterize the gaps between theoretical research on reflection and interface features in current apps, and offer suggestions about how reflection could be better supported.
Human-Centered Explainable AI (HCXAI): Beyond Opening the Black-Box of AI

Explainability of AI systems is crucial to hold them accountable because they are increasingly becoming consequential in our lives by powering high-stakes decisions in domains like healthcare and law. When it comes to Explainable AI (XAI), understanding who interacts with the black-box of AI is just as important as “opening” it, if not more. Yet the discourse of XAI has been predominantly centered around the black-box, suffering from deficiencies in meeting user needs and exacerbating issues of algorithmic opacity. To address these issues, researchers have called for human-centered approaches to XAI. In this second CHI workshop on Human-centered XAI (HCXAI), we build on the success of the first installment from CHI 2021 to expand the conversation around XAI. We chart the domain and shape the HCXAI discourse with reflective discussions from diverse stakeholders. The goal of the second installment is to go beyond the black box and examine how human-centered perspectives in XAI can be operationalized at the conceptual, methodological, and technical levels. Encouraging holistic (historical, sociological, and technical) approaches, we put an emphasis on “operationalizing”, aiming to produce actionable frameworks, transferable evaluation methods, concrete design guidelines, and articulate a coordinated research agenda for XAI.
CoreBug: Improving Effort-Aware Bug Prediction in Software Systems Using Generalized k-Core Decomposition in Class Dependency Networks

Complex network theory has been successfully introduced into the field of software engineering. Many works in the literature have built complex networks in software, usually called software networks, to represent software structure. Such software networks and their related graph algorithms have been proved effective in predicting software bugs. However, the software networks used were unweighted and undirected, neglecting the strength and direction of the couplings. Worse still, they ignored many important types of couplings between classes, such as local variable, instantiates, and access. All of these greatly affect the accuracy of the software network in representing the topological detail of software projects and ultimately affect the metrics derived from it. In this work, an improved effort-aware bug prediction approach named CoreBug is proposed. First, CoreBug uses a weighted directed class dependency network (WDCDN) to precisely describe classes and their couplings, including nine coupling types and their different coupling strengths and directions. Second, a generalized k-core decomposition is introduced to compute the coreness of each class in the WDCDN. Third, CoreBug combines the coreness of each class with its relative risk, as returned by the logistic regression, to quantify the risk of a given class being buggy. Empirical results on eighteen Java projects show that CoreBug is superior to the state-of-the-art approaches according to the average ranking of the Friedman test.
Resource Optimized Selective Image Encryption of Medical Images Using Multiple Chaotic Systems

Medical images typically contain personal and diagnostic information about patients and are typically shared among hospitals, doctors, and patients via public networks. As a result, they must be kept secure while being stored and transported to protect the patient's privacy. However, due to the unique qualities of digital image data, such as significant redundancy and correlation between pixels and their huge size, traditional cryptographic techniques are ineffective for providing proper security while encrypting them. The necessity to develop enhanced and specialized image encryption algorithms arise as traditional techniques cease to be a trustworthy option. This problem motivated them to develop many low computational complexity methods to encrypt these medical images. In this research work, Block Cipher based Region of interest medical image encryption with multiple maps is proposed. Primarily, Region of Interest (ROI) regions are extracted with the help of a Laplacian edge detection operator, and then based on the edges in the blocks, the image is classified into wanted (ROI) and unwanted (RONI or ROB) parts. Next, the important ROI regions are permuted circularly with the help of Arnold's cat map and angle value. Then permuted ROI part is encrypted using the duffling system and unimportant regions are unaltered. Finally, the ROI encrypted image is obtained by concating the ROI encrypted part and the Region of Background (ROB) unaltered part. The encryption algorithm proposed has better security performance which is demonstrated in the experiments such as attacks on data loss, differential attacks, statistical attacks, and brute force attacks. These results show that the security in the proposed method is much better than many chaotic encryption algorithms proposed in recent times.
Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models

Federated learning (FL) is a promising way to use the computing power of mobile devices while maintaining the privacy of users. Current work in FL, however, makes the unrealistic assumption that the users have ground-truth labels on their devices, while also assuming that the server has neither data nor labels. In this work, we consider the more realistic scenario where the users have only unlabeled data, while the server has some labeled data, and where the amount of labeled data is smaller than the amount of unlabeled data. We call this learning problem semi-supervised federated learning (SSFL). For SSFL, we demonstrate that a critical issue that affects the test accuracy is the large gradient diversity of the models from different users. Based on this, we investigate several design choices. First, we find that the so-called consistency regularization loss (CRL), which is widely used in semi-supervised learning, performs reasonably well but has large gradient diversity. Second, we find that Batch Normalization (BN) increases gradient diversity. Replacing BN with the recently-proposed Group Normalization (GN) can reduce gradient diversity and improve test accuracy. Third, we show that CRL combined with GN still has a large gradient diversity when the number of users is large. Based on these results, we propose a novel grouping-based model averaging method to replace the FedAvg averaging method. Overall, our grouping-based averaging, combined with GN and CRL, achieves better test accuracy than not just a contemporary paper on SSFL in the same settings (>10%), but also four supervised FL algorithms.
Immersive virtual reality for increasing presence and empathy

Abstract Background Immersive VR is still rarely used as an intervention for meeting the affective end goals of student learning despite its positive impact on affection. Also, studies regarding the use of immersive VR as an intervention for affective achievement in broader educational contexts are still lacking. Objectives: This study aimed to examine the effect of immersive VR and perspective‐taking on presence and empathy. Methods A total of 148 pre‐service teachers participated in experiments, using either a head‐mounted display or a flat screen device to view two VR videos with different perspective‐taking affordances. This study used a mixed design with one between‐subject variable of immersion level and one within‐subject variable of perspective‐taking to explore how immersive VR experiences influenced participants’ perceived level of presence and empathy. Results and Conclusions The results showed that the level of immersion affects perceived presence, but it was the type of perspective‐taking that affects empathetic reactions. We also found an interaction effect between immersion levels and perspective‐taking. The direct embodiment in VR combined with high immersion produced stronger empathy than with low immersion, while the perspective of an observer was better in evoking empathy when experienced with low immersion. Implications This study gives a guidance on how to take advantage of this new technology in educational settings, and apply it to instructional activities to enhance students’ empathy. In addition, it could serve as a reference when developing or introducing educational contents with respect to the types of contents that are more effective in educational settings.
Deep Neural Network With Consistency Regularization of Multi-Output Channels for Improved Tumor Detection and Delineation

Deep learning is becoming an indispensable tool for imaging applications, such as image segmentation, classification, and detection. In this work, we reformulate a standard deep learning problem into a new neural network architecture with multi-output channels, which reflects different facets of the objective, and apply the deep neural network to improve the performance of image segmentation. By adding one or more interrelated auxiliary-output channels, we impose an effective consistency regularization for the main task of pixelated classification (i.e., image segmentation). Specifically, multi-output-channel consistency regularization is realized by residual learning via additive paths that connect main-output channel and auxiliary-output channels in the network. The method is evaluated on the detection and delineation of lung and liver tumors with public data. The results clearly show that multi-output-channel consistency implemented by residual learning improves the standard deep neural network. The proposed framework is quite broad and should find widespread applications in various deep learning problems.
Self-Supervised Monocular Depth Estimation With Multiscale Perception

Extracting 3D information from a single optical image is very attractive. Recently emerging self-supervised methods can learn depth representations without using ground truth depth maps as training data by transforming the depth prediction task into an image synthesis task. However, existing methods rely on a differentiable bilinear sampler for image synthesis, which results in each pixel in a synthetic image being derived from only four pixels in the source image and causes each pixel in the depth map to perceive only a few pixels in the source image. In addition, when calculating the photometric error between a synthetic image and its corresponding target image, existing methods only consider the photometric error within a small neighborhood of each single pixel and therefore ignore correlations between larger areas, which causes the model to tend to fall into the local optima for small patches. In order to extend the perceptual area of the depth map over the source image, we propose a novel multi-scale method that downsamples the predicted depth map and performs image synthesis at different resolutions, which enables each pixel in the depth map to perceive more pixels in the source image and improves the performance of the model. As for the locality of photometric error, we propose a structural similarity (SSIM) pyramid loss to allow the model to sense the difference between images in multiple areas of different sizes. Experimental results show that our method achieves superior performance on both outdoor and indoor benchmarks.
Bayesian Feed Forward Neural Network-Based Efficient Anomaly Detection from Surveillance Videos

Automatic anomaly activity detection is difficult in video surveillance applications due to variations in size, type, shape, and objects' location.The traditional anomaly detection and classification methods may affect the overall segmentation accuracy.It requires the working groups to judge their constant attention if the captured activities are anomalous or suspicious.Therefore, this defect creates the need to automate this process with high accuracy.In addition to being extraordinary or questionable, the display does not contain the necessary recording frame and activity standard to help the quick judgment of the parts' specialized action.Therefore, to reduce the wastage of time and labor, this work utilizes a Bayesian Feed-Forward Neural Network (BFFN) algorithm to automate the anomaly Recognition System.It aims to detect irregularities, regular signs of aggression, and violence automatically, rather than the conventional methods.This work uses Bayesian Feed Forward Neural Networks to identify and classify frames with high activity levels.From there, the proposed system can issue detection alerts for threats indicating suspicious activity at a particular moment.The simulation results show that the proposed BFFN based anomaly detection responds better than existing methods.The sensitivity, specificity, accuracy, and F1-score of the proposed system are 98.69%, 97.01%, 99.65%, and 0.98%.
Future landscape visualization using a city digital twin: integration of augmented reality and drones with implementation of 3D model-based occlusion handling

Abstract The integration of augmented reality and drones allows past and future landscapes to be visualized from an aerial perspective. However, these visualizations still suffer from the occlusion problem, where the three-dimensional (3D) virtual model displayed in the real world is in front of a real-world object. Currently, city digital twins are essential for the sustainable development of cities and the development of detailed 3D models of cities. By visualizing the city digital twin, augmented reality can facilitate the participation of nonexpert citizens in the decision-making process of urban design, but research examples are limited. Here, using detailed city 3D models, we develop a digital-twin approach to outdoor augmented reality with occlusion handling for both first-person and bird’s-eye views. In a verification experiment, the occlusion handling accuracy of the prototype system was measured to be about 0.8 using intersection over union. The frame rate of the entire prototype system was about 30 fps, and the delay between the controller and the augmented reality device was about 3 s. The internet-based system architecture was developed to integrate augmented reality and drone systems. Our system allows multiple stakeholders involved in building construction projects to observe aerial perspectives of those projects, both on-site and off-site via an internet browser, using augmented reality with occlusion handling.
A Sliding Window Variational Outlier-Robust Kalman Filter Based on Student’s <i>t</i>-Noise Modeling

Existing robust state estimation methods are generally unable to distinguish model uncertainties (state outliers) from measurement outliers as they only exploit the current measurement. In this article, the measurements in a sliding window are, therefore, utilized to better distinguish them, and an adaptive method is embedded, leading to a sliding window variational outlier-robust Kalman filter based on Student’s <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</i> -noise modeling. Target tracking simulations and experiments show that the tracking accuracy and consistency of the proposed filter are superior to those of the existing state-of-the-art outlier-robust methods thanks to the improved ability to identify the outliers but at a cost of greater computational burden.
Locating and Counting Heads in Crowds With a Depth Prior

To simultaneously estimate the number of heads and locate heads with bounding boxes, we resort to detection-based crowd counting by leveraging RGB-D data and design a dual-path guided detection network (DPDNet). Specifically, to improve the performance of detection-based approaches for dense/tiny heads, we propose a density map guided detection module, which leverages density map to improve the head/non-head classification in detection network where the density implies the probability of a pixel being a head, and a depth-adaptive kernel that considers the variances in head sizes is also introduced to generate high-fidelity density map for more robust density map regression. In order to prevent dense heads from being filtered out during post-processing, we utilize such a density map for post-processing of head detection and propose a density map guided NMS strategy. Meanwhile, to improve the ability of detecting small heads, we also propose a depth-guided detection module to generate a dynamic dilated convolution to extract features of heads of different scales, and a depth-aware anchor is further designed for better initialization of anchor sizes in the detection framework. Then we use the bounding boxes whose sizes are generated with depth to train our DPDNet. Considering that existing RGB-D datasets are too small and not suitable for performance evaluation of data-driven based approaches, we collect two large-scale RGB-D crowd counting datasets, which comprise a synthetic dataset and a real-world dataset, respectively. Since the depth value at long-distance positions cannot be obtained in the real-world dataset, we further propose a depth completion method with meta learning, which fully utilizes the synthetic depth data to complete the depth value at long-distance positions. Extensive experiments on our proposed two RGB-D datasets and the MICC RGB-D counting dataset show that our method achieves the best performance for RGB-D crowd counting and localization. Further, our method can be easily extended to RGB image based crowd counting and achieves comparable or even better performance on the RGB datasets for both head counting and localization.
Self-Adaptive Particle Swarm Optimization-Based Echo State Network for Time Series Prediction

Echo state networks (ESNs), belonging to the family of recurrent neural networks (RNNs), are suitable for addressing complex nonlinear tasks due to their rich dynamic characteristics and easy implementation. The reservoir of the ESN is composed of a large number of sparsely connected neurons with randomly generated weight matrices. How to set the structural parameters of the ESN becomes a difficult problem in practical applications. Traditionally, the design of the parameters of the ESN structure is performed manually. The manual adjustment of the ESN parameters is not convenient since it is an extremely challenging and time-consuming task. This paper proposes an ensemble of five particle swarm optimization (PSO) strategies to design the structure of ESN and then reduce the manual intervention in the design process. An adaptive selection mechanism is used for each particle in the evolution to select a strategy from the strategy candidate pool for evolution. In addition, leaky integration neurons are used as reservoir internal neurons, which are added within the adaptive mechanism for optimization. The root mean squared error (RMSE) is adopted as the evaluation criterion. The experimental results on Mackey-Glass time series benchmark dataset show that the proposed method outperforms other traditional evolutionary methods. Furthermore, experimental results on electrocardiogram dataset show that the proposed method on the ensemble of PSO displays an excellent performance on real-world problems.
MLS-ABAC: Efficient Multi-Level Security Attribute-Based Access Control scheme

Realizing access control to sensitive data offloaded to a Cloud is challenging in the Internet of Things, where various devices with low computational power and different security levels are interconnected. Despite various solutions, the National Institute of Standards and Technology (NIST)'s Attribute-Based Access Control (ABAC) model is one of the preferred techniques in the literature. In this model, users who satisfy access policies using both static and dynamic attributes are allowed to access the data. However, NIST's ABAC model does not support encryption and therefore does not satisfy data confidentiality. Attribute-Based Encryption (ABE) is a known cryptographic primitive that enables fine-grained access control over encrypted data. However, currently the existing ABE schemes do not meet NIST's ABAC requirements or are not computationally efficient enough for IoT applications. In this paper, we propose a Multi-Level Security ABAC (MLS-ABAC) scheme that satisfies the requirements of NIST's ABAC model. Our construction is efficient and relies on a decryption outsourceable Ciphertext-Policy ABE scheme. Additionally, based on realistic application scenarios, only the authorized data users can decrypt the ciphertext, and check the integrity of the retrieved message. Furthermore, we present both conceptual and formal models for our proposed MLS-ABAC architecture along with performance metrics. The experimental results show that the proposed MLS-ABAC achieves a constant ciphertext size of ∼230 bytes and with encryption and decryption running times of ∼18 and ∼10 ms, respectively, independent of the number of attributes.