Reviewerly/exHarmony huggingface dataset:
Imbalanced Graph Classification via Graph-of-Graph Neural Networks

Graph Neural Networks (GNNs) have achieved unprecedented success in identifying categorical labels of graphs. However, most existing graph classification problems with GNNs follow the protocol of balanced data splitting, which misaligns with many real-world scenarios in which some classes have much fewer labels than others. Directly training GNNs under this imbalanced scenario may lead to uninformative representations of graphs in minority classes, and compromise the overall classification performance, which signifies the importance of developing effective GNNs towards handling imbalanced graph classification. Existing methods are either tailored for non-graph structured data or designed specifically for imbalanced node classification while few focus on imbalanced graph classification. To this end, we introduce a novel framework, Graph-of-Graph Neural Networks (G2GNN), which alleviates the graph imbalance issue by deriving extra supervision globally from neighboring graphs and locally from stochastic augmentations of graphs. Globally, we construct a graph of graphs (GoG) based on kernel similarity and perform GoG propagation to aggregate neighboring graph representations. Locally, we employ topological augmentation via masking node features or dropping edges with self-consistency regularization to generate stochastic augmentations of each graph that improve the model generalizability. Extensive graph classification experiments conducted on seven benchmark datasets demonstrate our proposed G2GNN outperforms numerous baselines by roughly 5% in both F1-macro and F1-micro scores.
Automated object detection on aerial images for limited capacity embedded device using a lightweight CNN model

With the growing demand for geospatial data, challenging aerial images with high spatial, spectral, and temporal resolution achieve excellent development. Currently, deep Convolutional Neural Network (CNN) structures are applied widely for object detection. Nevertheless, existing deep CNN-based models consist of complex network structures and require immense amounts of graphics processing unit (GPU) computation power with high energy consumption. Thus, achieving efficient real-time object detection for limited memory and processing capacity embedded device is a major challenge. This paper proposes a feasible and lightweight object detection model based on deep CNN where a mobile inverted bottleneck module is adopted in the backbone structure. Moreover, an enhanced spatial pyramid pooling is adopted to increase the receptive field in the network by concatenating the multi-scale local region features. The experimental results demonstrated that the proposed model achieved higher average precision and required the smallest memory storage compared to previous works. Moreover, the proposed model offers the best trade-offs in terms of detection accuracy, model size, and detection time which has excellent potential to be deployed on limited capacity embedded device.
On the Future of Cloud Engineering

Ever since the commercial offerings of the Cloud started appearing in 2006, the landscape of cloud computing has been undergoing remarkable changes with the emergence of many different types of service offerings, developer productivity enhancement tools, and new application classes as well as the manifestation of cloud functionality closer to the user at the edge. The notion of utility computing, however, has remained constant throughout its evolution, which means that cloud users always seek to save costs of leasing cloud resources while maximizing their use. On the other hand, cloud providers try to maximize their profits while assuring service-level objectives of the cloud-hosted applications and keeping operational costs low. All these outcomes require systematic and sound cloud engineering principles. The aim of this paper is to highlight the importance of cloud engineering, survey the landscape of best practices in cloud engineering and its evolution, discuss many of the existing cloud engineering advances, and identify both the inherent technical challenges and research opportunities for the future of cloud computing in general and cloud engineering in particular.
Abnormal event detection by a weakly supervised temporal attention network

Abnormal event detection aims to automatically identify unusual events that do not comply with expectation. Recently, many methods have been proposed to obtain the temporal locations of abnormal events under various determined thresholds. However, the specific categories of abnormal events are mostly neglect, which are important to help in monitoring agents to make decisions. In this study, a Temporal Attention Network (TANet) is proposed to capture both the specific categories and temporal locations of abnormal events in a weakly supervised manner. The TANet learns the anomaly score and specific category for each video segment with only video-level abnormal event labels. An event recognition module is exploited to predict the event scores for each video segment while a temporal attention module is proposed to learn a temporal attention value. Finally, to learn anomaly scores and specific categories, three constraints are considered: event category constraint, event separation constraint and temporal smoothness constraint. Experiments on the University of Central Florida Crime dataset demonstrate the effectiveness of the proposed method.
Multi-Person Extreme Motion Prediction

Human motion prediction aims to forecast future poses given a sequence of past 3D skeletons. While this problem has recently received increasing attention, it has mostly been tackled for single humans in isolation. In this paper, we explore this problem when dealing with humans performing collaborative tasks, we seek to predict the future motion of two interacted persons given two sequences of their past skeletons. We propose a novel cross interaction attention mechanism that exploits historical information of both persons, and learns to predict cross dependencies between the two pose sequences. Since no dataset to train such interactive situations is available, we collected ExPI (Extreme Pose Interaction) dataset, a new lab-based per-son interaction dataset of professional dancers performing Lindy-hop dancing actions, which contains 115 sequences with 30K frames annotated with 3D body poses and shapes. We thoroughly evaluate our cross interaction network on ExPI and show that both in short- and long-term predictions, it consistently outperforms state-of-the-art methods for single-person motion prediction. Our code and dataset are available at: https://team.inria.fr/robotlearn/multi-person-extreme-motionprediction/
Train++: An Incremental ML Model Training Algorithm to Create Self-Learning IoT Devices

The majority of Internet of Things (IoT) devices are tiny embedded systems with a micro-controller unit (MCU) as its brain. The memory footprint (SRAM, Flash, and EEPROM) of such MCU-based devices is often very limited, restricting onboard Machine Learning (ML) model training for large trainsets with high feature dimensions. To cope with memory issues, the current edge analytics approaches train high-quality ML models on the cloud GPUs (uses large volume historical data), then deploy the deep optimized version of the resultant models on edge devices for inference. Such approaches are inefficient in concept drift situations where the data generated at the device level vary frequently, and trained models are clueless on how to behave if previously unseen data arrives. In this paper, we present Train++, an incremental training algorithm that trains ML models locally at the device level (e.g., on MCUs and small CPUs) using the full n-samples of high-dimensional data. Train++ transforms even the most resource-constrained MCU-based IoT edge devices into intelligent devices that can locally build their own knowledge base on-the-fly using the live data, thus creating smart self-learning and autonomous problem-solving devices. Train++ algorithm is extensively evaluated on 5 popular MCU-boards, using 7 datasets of varying sizes and feature dimensions. A few exciting findings when analyzing the evaluation results are: (i) The proposed method reduces the onboard binary classifier training time by ≈ 10 - 226 sec across various commodity MCUs; (ii) Train++ can infer on MCUs for the entire test set in real-time of 1 ms; (iii) The accuracy improved by 5.15 - 7.3% since the incremental characteristic of Train++ enabled the loading of full n-samples of the high-dimensional datasets even on MCUs with only a few hundred kBs of memory.
Blockchain-Assisted Conditional Anonymity Privacy-Preserving Public Auditing Scheme With Reward Mechanism

In real-world scenarios, in order to encourage one to report others crimes, judicial department usually rents independent cloud storage spaces to receive the precious evidences from whistleblowers. Since the uploaded data are not controlled by cloud users, remote data integrity is very important. Public cloud auditing enables an auditor to periodically check the integrity of outsourcing data on behalf of users, without retrieving the entire data file. However, most existing data auditing schemes have potential security vulnerabilities, and thus cannot defense many security attacks (e.g., the man-in-the-middle attack). Meanwhile, it is significant to protect whistleblower’s identity privacy, reward the real data uploader, and further trace the responsibility of slanders accurately. From the aforementioned requirements, we present an efficient blockchain-assisted conditional anonymity privacy-preserving public auditing (BA-CAPPPA) scheme with reward mechanism. The Ethereum blockchain is integrated into BA-CAPPPA to enhance the security level of the whole public auditing mechanism. Theoretical analysis results show that the BA-CAPPPA achieves man-in-the-middle attack resistance, storage correctness guarantee, data privacy-preservation, conditional identity anonymity, and reward mechanism. Performance evaluations and comparisons demonstrate that BA-CAPPPA could outperform some state-of-the-art data auditing schemes.
Factors Influencing the Acceptance of Mobile Collaborative Learning for the Continuous Professional Development of Teachers

The COVID-19 epidemic has not only triggered a worldwide health catastrophe, it has also affected learning processes all over the world. While COVID-19 was in its initial wave of transmission, educational systems around the world witnessed bitter experiences in the form of the closure of institutes or a shift to distance (or online) learning techniques. The delivery of quality education was affected the most where the educational system was already weak due to the unavailability of effective teachers’ professional development programs and the lack of the integration of technology. This study extends the conventional continuous professional development (CPD) model of in-service teachers adopted in the province of Sindh, Pakistan. The extensions are based on the integration of mobile technology and support for collaborative learning and ongoing assistance, which are missed in the current traditional model. To assess the acceptance and use of the app, an Mobile collaborative learning (MCL) based CPD acceptance framework is designed by combining chosen components from a generally recognized Unified theory of acceptance and use of technology (UTAUT2) and other well-recognized acceptance models for the acceptance and use of the technology. In the context of CPD, a total of 14 constructs have been selected which include performance expectancy; effort expectancy; facilitating conditions; hedonic motivation; mobility; knowledge acquisition; content and information quality; on-the-job support; reward; behavioral intention; engagement; interactivity with peer &amp; experts; self-management and collaborative learning. Based on these constructs; a total of 13 hypotheses were developed with 55 measurement items. A total of 494 teachers participated in a two-week training session using the designed app. Based on the activity, a questionnaire was designed and distributed among the participants using WhatsApp groups during the academic year of 2021–2022. Out of 494 participants; 472 participants correctly filled in the questionnaires, which were selected for analysis. The data was analyzed using IBM SPSS version 21; and, for Structural Equation Modeling (SEM) analysis, IBM AMOS version 21 was used. According to the results, all thirteen hypotheses were found statistically significant showing the positive impact on the acceptance and use of the CPD app. The results of this study promote teachers’ and experts’ engagement and participation in CPD activities which not only improve the knowledge and skills of the learners but also impact their long-term professional development and success. This research may help the teachers’ training and related institutions to make better choices and develop a mobile collaborative learning system for school-based teacher support and training programs. As a basis, this study may also be used in other provinces of Pakistan and beyond to make it a CPD standard.
Machine and deep learning amalgamation for feature extraction in Industrial Internet-of-Things

In this paper, we develop a feature extraction model using the amalgamation of machine and deep learning techniques for Industrial Internet of Thing (IIoT). We train the model with the most effective feature set evaluated using machine learning algorithms and deep learning feature extraction methods. We test these features with deep learning based network models for validation. We consider error metrics and accuracy as the major factors for combining the machine learning and deep learning techniques. The error rates are analysed using mean square error which show low error rate for the subset than the model tested for full dataset. Further, mean square error, accuracy rate and false values are analysed to test the performance of the proposed model. The comparative analysis with IIoT dataset and existing methods show that our approach is 25% better than the others. • We address the multi-dimensional problem of the IIoT features to make an adequate feature extraction method. • We use machine learning classifier and wrapper subset evaluation technique with Naive-Bayes and meta bagging methods for feature selection. Deep learning auto encoders are implemented as a part of dimensionality reduction techniques and provides a amalgamation for the appropriateness of the features. We verify them with Principal Component Analysis (PCA) and ranking. Thus, only relevant features are extracted for analysis. • We create network models for analysing detection using deep learning Feed forward neural network and Elman Back propagation techniques for classification and detection with various training methods.
From “Cool Observer” to “Emotional Participant”: The Practice of Immersive Journalism

(2021). From "Cool Observer" to "Emotional Participant": The Practice of Immersive Journalism. Journalism Studies: Vol. 22, Journalism and Emotional Work - Guest Edited By Mervi Pantti and Karin Wahl-Jorgensen, pp. 1648-1664.
Comprehensive Review of Machine Learning (ML) in Image Defogging: Taxonomy of Concepts, Scenes, Feature Extraction, and Classification techniques

Images captured through a visual sensory system are degraded in a foggy scene, which negatively influences recognition, tracking, and detection of targets. Efficient tools are needed to detect, pre-process, and enhance foggy scenes. Machine learning (ML) has a significant role in image defogging domain for tackling adverse issues. Unfortunately, regardless of contributions that were made by ML, little attention has been attributed to this topic. This paper summarizes the role of ML methods and relevant aspects in the image defogging research area. Also, the basic terms and concepts are highlighted in image defogging topic. Feature extraction approaches with a summary of advantages and disadvantages are described. ML algorithms are also summarized that have been used for applications related to image defogging, that is, image denoising, image quality assessment, image segmentation, and foggy image classification. Open datasets are also discussed. Finally, the existing problems of the image defogging domain in general and, specifically related to ML which need to be further studied are discussed. To the best knowledge, this the first review paper which sheds a light on the role of ML and relevant aspects in the image defogging domain.
Quaternion Capsule Neural Network With Region Attention for Facial Expression Recognition in Color Images

Facial expression recognition (FER) is an essential subject of computer vision and human-computer interaction. It has been reported that many factors are closely related to the FER performance such as the pose, facial muscle variations, and the ignored color information in facial images. In this study, we propose a quaternion capsule neural network (Q-CapsNet) with region attention mechanism for FER in color images. The proposed Q-CapsNet is an end-to-end deep learning framework, which adopts the concept of quaternion theory to the Capsule Neural Network and further uses an attention mechanism to focus on the facial region of interests (ROIs). The Q-CapsNet addresses the FER problems in the following aspects. First, the internal dependencies between color channels are captured by the quaternion technique, which is of great importance in color image processing. Second, the Q-CapsNet could disentangle latent geometry information from facial images by quaternion capsules and quaternion routing algorithm. Third, inspired by the fact that facial expressions are mainly determined by several vital facial regions, the region attention mechanism is introduced to extract emotional features from the ROIs of the face. The proposed Q-CapsNet is evaluated on four public color FER datasets including MMI, Oulu-CASIA, RAF-DB, and SFEW. The comprehensive experiments and visualization results demonstrate that the proposed network outperforms comparative models and the state-of-the-art FER methods.
Semantic Structure in Deep Learning

Deep learning has recently come to dominate computational linguistics, leading to claims of human-level performance in a range of language processing tasks. Like much previous computational work, deep learning–based linguistic representations adhere to the distributional meaning-in-use hypothesis, deriving semantic representations from word co-occurrence statistics. However, current deep learning methods entail fundamentally new models of lexical and compositional meaning that are ripe for theoretical analysis. Whereas traditional distributional semantics models take a bottom-up approach in which sentence meaning is characterized by explicit composition functions applied to word meanings, new approaches take a top-down approach in which sentence representations are treated as primary and representations of words and syntax are viewed as emergent. This article summarizes our current understanding of how well such representations capture lexical semantics, world knowledge, and composition. The goal is to foster increased collaboration on testing the implications of such representations as general-purpose models of semantics.
3D pose estimation and future motion prediction from 2D images

This paper considers to jointly tackle the highly correlated tasks of estimating 3D human body poses and predicting future 3D motions from RGB image sequences. Based on Lie algebra pose representation, a novel self-projection mechanism is proposed that naturally preserves human motion kinematics. This is further facilitated by a sequence-to-sequence multi-task architecture based on an encoder-decoder topology, which enables us to tap into the common ground shared by both tasks. Finally, a global refinement module is proposed to boost the performance of our framework. The effectiveness of our approach, called PoseMoNet, is demonstrated by ablation tests and empirical evaluations on Human3.6M and HumanEva-I benchmark, where competitive performance is obtained comparing to the state-of-the-arts.
Human Complex Activity Recognition With Sensor Data Using Multiple Features

In order to achieve better results of human complex activity recognition, the optimal feature representation of human complex activities from sensor data are studied. Features from multiple sets that are obtained by the learned networks, the manual extraction, and the location information are fused to generate mixed features to achieve better results of human complex activity recognition. A method extracting multi-layer features from the hybrid CNN and BLSTM network is proposed. The output feature vector of the second layer of the CNN and that of the BLSTM are combined to generate multi-layer features. Meanwhile, a new feature selection method based on SFS and network weight analysis is proposed. The method is to select a series of features extracted from the segmented sensor data using manual methods as dominant features. The location information of complex activities after one-hot encoding is also used as feature, which is then fused with multi-layer features and manual dominant features to generate mixed features. To further improve the recognition performance, the sensor data collected at different body positions are separated and used independently to train the hybrid CNN and BLSTM network to obtain their respective state information. The PAMAP2 and UT-Data datasets are used in our experiments to verify the proposed method. The results show that the method based on multiple features proposed in this paper dramatically outperforms the existing state-of-the-art methods for human complex activity recognition using sensor data.
Adaptive Spatiotemporal Representation Learning for Skeleton-Based Human Action Recognition

How do humans recognize an action or an interaction in the real world? Due to the diversity of viewing perspectives, it is a challenge for humans to identify a regular activity when they observe it from an uncommon perspective. We argue that discriminative spatiotemporal information remains an essential cue for human action recognition. Most existing skeleton-based methods learn optimal representation based on the human-crafted criterion that requires many labeled data and much human effort. This article introduces adaptive skeleton-based neural networks to learn optimal spatiotemporal representation automatically through a data-driven manner. First, an adaptive skeleton representation transformation method (ASRT) is proposed to model view-variation data without hand-crafted criteria. Next, powered by a novel attentional LSTM (C3D-LSTM) encapsulated with 3-D-convolution, the proposed model could effectively enable memory blocks to learn short-term frame dependency and long-term relations. Hence, the proposed model can more accurately understand long-term or complex actions. Furthermore, a data enhancement-driven end-to-end training scheme is proposed to train key parameters under fewer training samples. Enhanced by learned high-performance spatiotemporal representation, the proposed model achieves state-of-the-art performance on five challenging benchmarks.