Reviewerly/exHarmony huggingface dataset:
Reconstructive Sequence-Graph Network for Video Summarization

Exploiting the inner-shot and inter-shot dependencies is essential for key-shot based video summarization. Current approaches mainly devote to modeling the video as a frame sequence by recurrent neural networks. However, one potential limitation of the sequence models is that they focus on capturing local neighborhood dependencies while the high-order dependencies in long distance are not fully exploited. In general, the frames in each shot record a certain activity and vary smoothly over time, but the multi-hop relationships occur frequently among shots. In this case, both the local and global dependencies are important for understanding the video content. Motivated by this point, we propose a reconstructive sequence-graph network (RSGN) to encode the frames and shots as sequence and graph hierarchically, where the frame-level dependencies are encoded by long short-term memory (LSTM), and the shot-level dependencies are captured by the graph convolutional network (GCN). Then, the videos are summarized by exploiting both the local and global dependencies among shots. Besides, a reconstructor is developed to reward the summary generator, so that the generator can be optimized in an unsupervised manner, which can avert the lack of annotated data in video summarization. Furthermore, under the guidance of reconstruction loss, the predicted summary can better preserve the main video content and shot-level dependencies. Practically, the experimental results on three popular datasets (i.e., SumMe, TVsum and VTW) have demonstrated the superiority of our proposed approach to the summarization task.
CacheOut: Leaking Data on Intel CPUs via Cache Evictions

Recent transient-execution attacks, such as RIDL, Fallout, and ZombieLoad, demonstrated that attackers can leak information while it transits through microarchitectural buffers. Named Microarchitectural Data Sampling (MDS) by Intel, these attacks are likened to "drinking from the firehose", as the attacker has little control over what data is observed and from what origin. Unable to prevent the buffers from leaking, Intel issued countermeasures via microcode updates that overwrite the buffers when the CPU changes security domains.In this work we present CacheOut, a new microarchitectural attack that is capable of bypassing Intel’s buffer overwrite countermeasures. We observe that as data is being evicted from the CPU’s L1 cache, it is often transferred back to the leaky CPU buffers where it can be recovered by the attacker. CacheOut improves over previous MDS attacks by allowing the attacker to choose which data to leak from the CPU’s L1 cache, as well as which part of a cache line to leak. We demonstrate that CacheOut can leak information across multiple security boundaries, including those between processes, virtual machines, user and kernel space, and from SGX enclaves.
Dilated causal convolution with multi-head self attention for sensor human activity recognition

Abstract Systems of sensor human activity recognition are becoming increasingly popular in diverse fields such as healthcare and security. Yet, developing such systems poses inherent challenges due to the variations and complexity of human behaviors during the performance of physical activities. Recurrent neural networks, particularly long short-term memory have achieved promising results on numerous sequential learning problems, including sensor human activity recognition. However, parallelization is inhibited in recurrent networks due to sequential operation and computation that lead to slow training, occupying more memory and hard convergence. One-dimensional convolutional neural network processes input temporal sequential batches independently that lead to effectively executed operations in parallel. Despite that, a one-dimensional Convolutional Neural Network is not sensitive to the order of the time steps which is crucial for accurate and robust systems of sensor human activity recognition. To address this problem, we propose a network architecture based on dilated causal convolution and multi-head self-attention mechanisms that entirely dispense recurrent architectures to make efficient computation and maintain the ordering of the time steps. The proposed method is evaluated for human activities using smart home binary sensors data and wearable sensor data. Results of conducted extensive experiments on eight public and benchmark HAR data sets show that the proposed network outperforms the state-of-the-art models based on recurrent settings and temporal models.
A Unified View of Causal and Non-causal Feature Selection

In this article, we aim to develop a unified view of causal and non-causal feature selection methods. The unified view will fill in the gap in the research of the relation between the two types of methods. Based on the Bayesian network framework and information theory, we first show that causal and non-causal feature selection methods share the same objective. That is to find the Markov blanket of a class attribute, the theoretically optimal feature set for classification. We then examine the assumptions made by causal and non-causal feature selection methods when searching for the optimal feature set, and unify the assumptions by mapping them to the restrictions on the structure of the Bayesian network model of the studied problem. We further analyze in detail how the structural assumptions lead to the different levels of approximations employed by the methods in their search, which then result in the approximations in the feature sets found by the methods with respect to the optimal feature set. With the unified view, we can interpret the output of non-causal methods from a causal perspective and derive the error bounds of both types of methods. Finally, we present practical understanding of the relation between causal and non-causal methods using extensive experiments with synthetic data and various types of real-world data.
Visual, textual or hybrid: the effect of user expertise on different explanations

As the use of AI algorithms keeps rising continuously, so does the need for their transparency and accountability. However, literature often adopts a one-size-fits-all approach for developing explanations when in practice, the type of explanations needed depends on the type of end-user. This research will look at user expertise as a variable to see how different levels of expertise influence the understanding of explanations. The first iteration consists of developing two common types of explanations (visual and textual explanations) that explain predictions made by a general class of predictive model learners. These explanations are then evaluated by users of different expertise backgrounds to compare the understanding and ease-of-use of each type of explanation with respect to the different expertise groups. Results show strong differences between experts and lay users when using visual and textual explanations, as well as lay users having a preference for visual explanations which they perform significantly worse with. To solve this problem, the second iteration of this research focuses on the shortcomings of the first two explanations and tries to minimize the difference in understanding between both expertise groups. This is done through the means of developing and testing a candidate solution in the form of hybrid explanations, which essentially combine both visual and textual explanations. This hybrid form of explanations shows a significant improvement in terms of correct understanding (for lay users in particular) when compared to visual explanations, whilst not compromising on ease-of-use at the same time.
Decoupling Representation Learning and Classification for GNN-based Anomaly Detection

GNN-based anomaly detection has recently attracted considerable attention. Existing attempts have thus far focused on jointly learning the node representations and the classifier for detecting the anomalies. Inspired by the recent advances of self-supervised learning (SSL) on graphs, we explore another possibility of decoupling the node representation learning and the classification for anomaly detection. We conduct a preliminary study to show that decoupled training using existing graph SSL schemes to represent nodes can obtain performance gains over joint training, but it may deteriorate when the behavior patterns and the label semantics become highly inconsistent. To be less biased by the inconsistency, we propose a simple yet effective graph SSL scheme, called Deep Cluster Infomax (DCI) for node representation learning, which captures the intrinsic graph properties in more concentrated feature spaces by clustering the entire graph into multiple parts. We conduct extensive experiments on four real-world datasets for anomaly detection. The results demonstrate that decoupled training equipped with a proper SSL scheme can outperform joint training in AUC. Compared with existing graph SSL schemes, DCI can help decoupled training gain more improvements.
A comprehensive review on remote sensing image registration

Over the last two decades, there has been a significant improvement in the quality and volume of the remote-sensing images. With the extensive availability of the images, these are used in many applications such as change detection, image mosaicing, and image fusion where image registration is very necessary. However, image registration is a challenging task in remote sensing due to the following reasons: geometric differences between images, intensity differences, and noise affect. Over the years, wide variety of algorithms have come into existence to handle these issues. In this paper, a comprehensive review is presented on these remote sensing image registration methods. At first, the general information of remote sensing image registration, its classifications, and application areas are provided. Then, an exhaustive review is given on the existing methods and their merits as well as shortcomings are mentioned. Finally, a comparative analysis is provided for the state-of-the-art methods and current challenges of this research are presented.
Overcoming Noisy and Irrelevant Data in Federated Learning

Many image and vision applications require a large amount of data for model training. Collecting all such data at a central location can be challenging due to data privacy and communication bandwidth restrictions. Federated learning is an effective way of training a machine learning model in a distributed manner from local data collected by client devices, which does not require exchanging the raw data among clients. A challenge is that among the large variety of data collected at each client, it is likely that only a subset is relevant for a learning task while the rest of data has a negative impact on model training. Therefore, before starting the learning process, it is important to select the subset of data that is relevant to the given federated learning task. In this paper, we propose a method for distributedly selecting relevant data, where we use a benchmark model trained on a small benchmark dataset that is task-specific, to evaluate the relevance of individual data samples at each client and select the data with sufficiently high relevance. Then, each client only uses the selected subset of its data in the federated learning process. The effectiveness of our proposed approach is evaluated on multiple real-world image datasets in a simulated system with a large number of clients, showing up to 25% improvement in model accuracy compared to training with all data.
Fault Localization with Code Coverage Representation Learning

In this paper, we propose DeepRL4FL, a deep learning fault localization (FL) approach that locates the buggy code at the statement and method levels by treating FL as an image pattern recognition problem. DeepRL4FL does so via novel code coverage representation learning (RL) and data dependencies RL for program statements. Those two types of RL on the dynamic information in a code coverage matrix are also combined with the code representation learning on the static information of the usual suspicious source code. This combination is inspired by crime scene investigation in which investigators analyze the crime scene (failed test cases and statements) and related persons (statements with dependencies), and at the same time, examine the usual suspects who have committed a similar crime in the past (similar buggy code in the training data). For the code coverage information, DeepRL4FL first orders the test cases and marks error-exhibiting code statements, expecting that a model can recognize the patterns discriminating between faulty and non-faulty statements/methods. For dependencies among statements, the suspiciousness of a statement is seen taking into account the data dependencies to other statements in execution and data flows, in addition to the statement by itself. Finally, the vector representations for code coverage matrix, data dependencies among statements, and source code are combined and used as the input of a classifier built from a Convolution Neural Network to detect buggy statements/methods. Our empirical evaluation shows that DeepRL4FL improves the top-1 results over the state-of-the-art statement-level FL baselines from 173.1% to 491.7%. It also improves the top-1 results over the existing method-level FL baselines from 15.0% to 206.3%.
Augmented Reality in Education: An Overview of Twenty-Five Years of Research

Research on augmented reality (AR) in education is gaining momentum worldwide. This field has been actively growing over the past decades in terms of the research and development of new technologies. Reviews in the field of AR in education consist of systematic literature reviews and meta-analyses (around 45), surveys (around 33), and only one bibliometric analysis. However, these reviews do not provide a general synthesis of the research published in the field to depict its evolution over the years. This study used the metadata of articles from a 25-year period (1995-2020) to conduct a bibliometric analysis. A total of 3,475 studies were considered. In this study, we used tools such as the Scopus database, the bibliometrix R package, and the VOSviewer analysis tool. The analysis of the literature is based on the metadata, author, content, and citation information extracted from the dataset. In addition, we focus on comparing literature published mainly in journals (articles, articles in press, and reviews) and those published in other sources (conference papers, books, and book chapters). Practitioners could use the results of this study to make decisions about the adoption of AR technologies in education.
Evaluation of BERT and ALBERT Sentence Embedding Performance on Downstream NLP Tasks

Contextualized representations from a pre-trained language model are central to achieve a high performance on downstream NLP task. The pre-trained BERT and A Lite BERT (ALBERT) models can be fine-tuned to give state-of-the-art results in sentence-pair regressions such as semantic textual similarity (STS) and natural language inference (NLI). Although BERT-based models yield the [CLS] token vector as a reasonable sentence embedding, the search for an optimal sentence embedding scheme remains an active research area in computational linguistics. This paper explores on sentence embedding models for BERT and ALBERT. In particular, we take a modified BERT network with siamese and triplet network structures called Sentence-BERT (SBERT) and replace BERT with ALBERT to create Sentence-ALBERT (SALBERT). We also experiment with an outer CNN sentence-embedding network for SBERT and SALBERT. We evaluate performances of all sentence-embedding models considered using the STS and NLI datasets. The empirical results indicate that our CNN architecture improves ALBERT models substantially more than BERT models for STS benchmark. Despite significantly fewer model parameters, ALBERT sentence embedding is highly competitive to BERT in downstream NLP evaluations.
The Show Must Go On:

Co-designing with children in an online environment is increasingly important due to external factors, such as the COVID-19 pandemic, and the diversification and inclusion of youth participants. Many prior studies about co-design with youth focus on co-located or asynchronous online sessions. However, conducting synchronous online co-design sessions adds layers of complexity and uncertainty to collaboration. This paper introduces a model explicating factors to consider when co-designing with children synchronously in an online space. We examined ten consecutive intergenerational participatory design sessions online where children (ages 7-11) and adults designed new technologies. Along with highlighting unexpected moments and interactions, we use theories of improvisation to guide our understanding of dynamic situations that are out of the control of researchers. This work contributes to improving theoretical understanding of improvisation as a method of inquiry for co-designing with youth, and offers practical suggestions for suitable online co-design techniques and implementation.
A Recommendation Engine for Predicting Movie Ratings Using a Big Data Approach

In this era of big data, the amount of video content has dramatically increased with an exponential broadening of video streaming services. Hence, it has become very strenuous for end-users to search for their desired videos. Therefore, to attain an accurate and robust clustering of information, a hybrid algorithm was used to introduce a recommender engine with collaborative filtering using Apache Spark and machine learning (ML) libraries. In this study, we implemented a movie recommendation system based on a collaborative filtering approach using the alternating least squared (ALS) model to predict the best-rated movies. Our proposed system uses the last search data of a user regarding movie category and references this to instruct the recommender engine, thereby making a list of predictions for top ratings. The proposed study used a model-based approach of matrix factorization, the ALS algorithm along with a collaborative filtering technique, which solved the cold start, sparse, and scalability problems. In particular, we performed experimental analysis and successfully obtained minimum root mean squared errors (oRMSEs) of 0.8959 to 0.97613, approximately. Moreover, our proposed movie recommendation system showed an accuracy of 97% and predicted the top 1000 ratings for movies.
Comparing physiological responses during cognitive tests in virtual environments vs. in identical real-world environments

Abstract Immersive virtual environments (VEs) are increasingly used to evaluate human responses to design variables. VEs provide a tremendous capacity to isolate and readily adjust specific features of an architectural or product design. They also allow researchers to safely and effectively measure performance factors and physiological responses. However, the success of this form of design-testing depends on the generalizability of response measurements between VEs and real-world contexts. At the current time, there is very limited research evaluating the consistency of human response data across identical real and virtual environments. Rendering tools were used to precisely replicate a real-world classroom in virtual space. Participants were recruited and asked to complete a series of cognitive tests in the real classroom and in the virtual classroom. Physiological data were collected during these tests, including electroencephalography (EEG), electrocardiography (ECG), electrooculography (EOG), galvanic skin response (GSR), and head acceleration. Participants’ accuracy on the cognitive tests did not significantly differ between the real classroom and the identical VE. However, the participants answered the tests more rapidly in the VE. No significant differences were found in eye blink rate and heart rate between the real and VR settings. Head acceleration and GSR variance were lower in the VE setting. Overall, EEG frequency band-power was not significantly altered between the real-world classroom and the VE. Analysis of EEG event-related potentials likewise indicated strong similarity between the real-world classroom and the VE, with a single exception related to executive functioning in a color-mismatch task.
Analysis of Darknet Traffic for Criminal Activities Detection Using TF-IDF and Light Gradient Boosted Machine Learning Algorithm

Darkweb also called as sinkholes, blackholes, network telescopes, and darknet is the environment and the most favorable platform for illegal activities due to hidden IP address and therefore counted as unused address space, which is not available for normal user, and the anonymous behavior acts as catalyst for criminal or unauthorized behavior conduction. It is very difficult to suddenly trace the location of malicious activity origin but by traffic analysis and understanding the patterns, suspicious activities including email communication, audio–video streaming, chatting P2P, browsing data, chatting, and voice over Internet protocol constitute the hidden world web traffic. Several methods have been deployed to analysis and classify darkweb network traffic. The proposed work detects worms, dos attack, backdoor, DDos attack, RDoS attack, spam, and malicious contents. In the proposed work, term frequency-inverse document frequency (TF-IDF) and light gradient boosted machine algorithm method has been implemented on darknet traffic data. The light gradient boosted machine algorithm shows the value of 98.97% as accuracy and thus outperforms the other algorithms based on experiment values.
Breast mass classification with transfer learning based on scaling of deep representations

Ultrasound (US) imaging is widely used to help radiologists in diagnosing breast cancer. In this work, we propose a deep learning based approach to breast mass classification in US. Transfer learning with convolutional neural networks (CNNs) is commonly used to develop object recognition models in medical image analysis. The most widely used fine-tuning techniques aim to modify weights of pre-trained networks to address target medical problems. However, fine-tuning can be difficult when the number of trainable parameters of the pre-trained network is large and the available medical data are scarce. To address this issue, we propose a novel transfer learning technique based on deep representation scaling (DRS) layers, which are inserted between the blocks of a pre-trained CNN to enable better flow of information in the network. During network training, we only update the parameters of the DRS layers in order to adjust the pre-trained CNN to process breast mass US images. We present that the DRS based approach greatly reduces the number of trainable parameters, and achieves better or comparable performance to the standard transfer learning techniques. The proposed DRS layer method combined with the standard fine-tuning techniques achieved excellent breast mass classification performance, with area under the receiver operating characteristic curve of 0.955 and accuracy of 0.915.