Reviewerly/exHarmony huggingface dataset:
Graph-based High-order Relation Modeling for Long-term Action Recognition

Long-term actions involve many important visual concepts, e.g., objects, motions, and sub-actions, and there are various relations among these concepts, which we call basic relations. These basic relations will jointly affect each other during the temporal evolution of long-term actions, which forms the high-order relations that are essential for long-term action recognition. In this paper, we propose a Graph-based High-order Relation Modeling (GHRM) module to exploit the high-order relations in the long-term actions for long-term action recognition. In GHRM, each basic relation in the long-term actions will be modeled by a graph, where each node represents a segment in a long video. Moreover, when modeling each basic relation, the information from all the other basic relations will be incorporated by GHRM, and thus the high-order relations in the long-term actions can be well exploited. To better exploit the high-order relations along the time dimension, we design a GHRM-layer consisting of a Temporal-GHRM branch and a Semantic-GHRM branch, which aims to model the local temporal high-order relations and global semantic high-order relations. The experimental results on three long-term action recognition datasets, namely, Breakfast, Charades, and MultiThumos, demonstrate the effectiveness of our model.
Balancing sequential data to predict students at-risk using adversarial networks

Class imbalance is a challenging problem especially in a supervised learning setup, as most classification algorithms are designed for balanced class distributions. Although various up-sampling approaches exist for eliminating the class imbalance, however, they do not handle the complexities of sequential data. In this study, using the data of over 30,000 students from the Open University (UK), we implement a deep-learning-based approach using adversarial networks, Sequential Conditional Generative Adversarial Network (SC-GAN) that encapsulates the past behavior of each student for its previous sequences and generates synthetic student records for the next timestamp. The proposed approach is devised to generate instances, which are augmented with the actual data to eliminate class imbalance. A performance comparison of the proposed SC-GAN with the standard up-sampling methods is also presented and the results validate the proposed method with an improved AUC of 7.07% and 6.53%, respectively, when compared with conventional Random Over-sampling and Sythetic Minority Oversampling techniques.
Secure Video Surveillance Framework in Smart City

In the construction process of smart cities, more and more video surveillance systems have been deployed for traffic, office buildings, shopping malls, and families. Thus, the security of video surveillance systems has attracted more attention. At present, many researchers focus on how to select the region of interest (RoI) accurately and then realize privacy protection in videos by selective encryption. However, relatively few researchers focus on building a security framework by analyzing the security of a video surveillance system from the system and data life cycle. By analyzing the surveillance video protection and the attack surface of a video surveillance system in a smart city, we constructed a secure surveillance framework in this manuscript. In the secure framework, a secure video surveillance model is proposed, and a secure authentication protocol that can resist man-in-the-middle attacks (MITM) and replay attacks is implemented. For the management of the video encryption key, we introduced the Chinese remainder theorem (CRT) on the basis of group key management to provide an efficient and secure key update. In addition, we built a decryption suite based on transparent encryption to ensure the security of the decryption environment. The security analysis proved that our system can guarantee the forward and backward security of the key update. In the experiment environment, the average decryption speed of our system can reach 91.47 Mb/s, which can meet the real-time requirement of practical applications.
Deep Reinforcement Agent for Scheduling in HPC

Cluster scheduler is crucial in high-performance computing (HPC). It determines when and which user jobs should be allocated to available system resources. Existing cluster scheduling heuristics are developed by human experts based on their experience with specific HPC systems and workloads. However, the increasing complexity of computing systems and the highly dynamic nature of application workloads have placed tremendous burden on manually designed and tuned scheduling heuristics. More aggressive optimization and automation are needed for cluster scheduling in HPC. In this work, we present an automated HPC scheduling agent named DRAS (Deep Reinforcement Agent for Scheduling) by leveraging deep reinforcement learning. DRAS is built on a novel, hierarchical neural network incorporating special HPC scheduling features such as resource reservation and backfilling. A unique training strategy is presented to enable DRAS to rapidly learn the target environment. Once being provided a specific scheduling objective given by system manager, DRAS automatically learns to improve its policy through interaction with the scheduling environment and dynamically adjusts its policy as workload changes. The experiments with different production workloads demonstrate that DRAS outperforms the existing heuristic and optimization approaches by up to 45%.
Failure and Success in Using Mozilla Hubs for Online Teaching in a Movie Production Course

In the autumn 2020 the course Digital Movie Making was given in a fully online mode, due to the restrictions and lockdowns related to the covid-19 pandemic. With the intention of avoiding Zoom fatigue and provide a more creative and engaging online teaching environment, the social VR platform Mozilla Hubs was chosen for all the lectures, seminars and supervision. The two main reasons for choosing Mozilla Hubs were the openness of the platform, providing wide opportunities for creating and setting up your own virtual space, as well as the option to access the platform via a web browser. However, Mozilla Hubs has a number of usability and technical flaws, making it clumsy to use, and the initial course introduction and guest lecture suffered severe technical issues when all 25 students attended simultaneously. A decision was made to only use Mozilla Hubs for supervision, and these meetings with few students turned out successful. Based on the observations from these learning activities, a number of advantages and disadvantages with VR in general and Mozilla Hubs specifically is presented and discussed.
Improving Conversational Recommender System by Pretraining Billion-scale Knowledge Graph

Conversational Recommender Systems (CRSs) in E-commerce platforms aim to recommend items to users via multiple conversational interactions. Click-through rate (CTR) prediction models are commonly used for ranking candidate items. However, most CRSs are suffer from the problem of data scarcity and sparseness. To address this issue, we propose a novel knowledge-enhanced deep cross network (K-DCN), a two-step (pretrain and fine-tune) CTR prediction model to recommend items. We first construct a billion-scale conversation knowledge graph (CKG) from information about users, items and converations, and then pretrain CKG by introducing knowledge graph embedding method and graph convolution network to encode semantic and structural information respectively. To make the CTR prediction model sensible of current state of users and the relationship between dialogues and items, we introduce user-state and dialogue-interaction representations based on pre-trained CKG and propose K-DCN. In K-DCN, we fuse the user-state representation, dialogue-interaction representation and other normal feature representations via deep cross network, which will give the rank of candidate items to be recommended. We experimentally prove that our proposal significantly outperforms baselines and show it's real application in Alime.
Ego-Exo: Transferring Visual Representations from Third-person to First-person Videos

We introduce an approach for pre-training egocentric video models using large-scale third-person video datasets. Learning from purely egocentric data is limited by low dataset scale and diversity, while using purely exocentric (third-person) data introduces a large domain mismatch. Our idea is to discover latent signals in third-person video that are predictive of key egocentric-specific properties. Incorporating these signals as knowledge distillation losses during pre-training results in models that benefit from both the scale and diversity of third-person video data, as well as representations that capture salient egocentric properties. Our experiments show that our "Ego-Exo" framework can be seamlessly integrated into standard video models; it outperforms all baselines when fine-tuned for egocentric activity recognition, achieving state-of-the-art results on Charades-Ego and EPIC-Kitchens-100.
Multi-view learning methods with the LINEX loss for pattern classification

Multi-view learning concentrates on leveraging the consensus and complementarity information among multiple distinct feature representations to improve the performance. Most multi-view learning models deal with two main issues. Firstly, how to fully exploit the view-agreement and view-discrepancy poses a major challenge. Secondly, how to design a general multi-view model is indispensable. By inheriting the asymmetric merit of LINEX loss, we propose a general multi-view LINEX SVM framework, which includes two models called MVLSVM-CO and MVLSVM-SIM. They can not only use LINEX loss function to flexibly distinguish the error-prone samples of both classes, but also take advantage of the consistency and the complementarity of distinct views in multi-view scenario. An iterative two-step strategy is adopted to solve the optimization problems efficiently. Furthermore, we theoretically analyze the view-consistency and generalization capability of the proposed models by using Rademacher complexity. The extensive experiments confirm the effectiveness of MVLSVM-CO and MVLSVM-SIM.
Hierarchical Federated Learning for Collaborative IDS in IoT Applications

As the Internet-of-Things devices are being very widely adopted in all fields, such as smart houses, healthcare, and transportation, extremely huge amounts of data are being gathered, shared, and processed. This fact raises many challenges on how to make the best use of this amount of data to improve the IoT systems' security using artificial intelligence, with taking into consideration the resource limitations in IoT devices and issues regarding data privacy. Different techniques have been studied and developed throughout the years. For example, Federated Learning (FL), which is an emerging learning technique that is very well known for preserving and respecting the privacy of the collaborating clients' data during model training. Therefore, in this paper, the concepts of FL and Hierarchical Federated Learning (HFL) are evaluated and compared with respect of detection accuracy and speed of convergence, through simulating an Intrusion Detection System for Internet-of-Things applications. The imbalanced NSL-KDD dataset was used in this work. Despite its infrastructure overhead, HFL proved its superiority over FL in terms of training loss, testing accuracy, and speed of convergence in three study cases. HFL also showed its efficiency over FL in reducing the effect of the non-identically and independently (non-iid) distributed data on the collaborative learning process.
AdvDoor: adversarial backdoor attack of deep learning system

Deep Learning (DL) system has been widely used in many critical applications, such as autonomous vehicles and unmanned aerial vehicles. However, their security is threatened by backdoor attack, which is achieved by adding artificial patterns on specific training data. Existing attack methods normally poison the data using a patch, and they can be easily detected by existing detection methods. In this work, we propose the Adversarial Backdoor, which utilizes the Targeted Universal Adversarial Perturbation (TUAP) to hide the anomalies in DL models and confuse existing powerful detection methods. With extensive experiments, it is demonstrated that Adversarial Backdoor can be injected stably with an attack success rate around 98%. Moreover, Adversarial Backdoor can bypass state-of-the-art backdoor detection methods. More specifically, only around 37% of the poisoned models can be caught, and less than 29% of the poisoned data cannot bypass the detection. In contrast, for the patch backdoor, all the poisoned models and more than 80% of the poisoned data will be detected. This work intends to alarm the researchers and developers of this potential threat and to inspire the designing of effective detection methods.
Data Security in Cloud Computing Using Abe-Based Access Control

Business organizations and individual users are using cloud storage for storing their data and files. Cloud storage is managed by cloud service provider (CSP) being third party person to the data owners. Cloud storage consists of user's confidential data. After storing data in cloud, the owner of data cannot have control over data, where owner cannot trust the CSP because possibility of a malicious administrator. Based on this, different schemes are proposed. Security is a major concern for cloud stored data, and CSP has to provide trust to the data owner on security of the cloud stored data. In general, security to data and applications is provided through authentication and authorization. Security through authentication is provided by distributing user name and password to data users. However, the organizational user is not allowed to access all the organizational data. Authorization for accessing the data is provided by using access control models. Regular models are not enough to use the CSP based on the models uses dynamic method and proposed different models using attribute-based encryption (ABE). Earlier access control models cannot be used because of multiple disadvantages. This chapter will discuss dynamic access control model named as RA-HASBE. This model is proved to be scalable and flexible, due to sub-domain hierarchy. It is also proved to be dynamic by permitting user to access the data by risk evaluation using risk engine.
Air Learning: a deep reinforcement learning gym for autonomous aerial robot visual navigation

Abstract We introduce Air Learning, an open-source simulator, and a gym environment for deep reinforcement learning research on resource-constrained aerial robots. Equipped with domain randomization, Air Learning exposes a UAV agent to a diverse set of challenging scenarios. We seed the toolset with point-to-point obstacle avoidance tasks in three different environments and Deep Q Networks (DQN) and Proximal Policy Optimization (PPO) trainers. Air Learning assesses the policies’ performance under various quality-of-flight (QoF) metrics, such as the energy consumed, endurance, and the average trajectory length, on resource-constrained embedded platforms like a Raspberry Pi. We find that the trajectories on an embedded Ras-Pi are vastly different from those predicted on a high-end desktop system, resulting in up to $$40\%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow><mml:mn>40</mml:mn><mml:mo>%</mml:mo></mml:mrow></mml:math> longer trajectories in one of the environments. To understand the source of such discrepancies, we use Air Learning to artificially degrade high-end desktop performance to mimic what happens on a low-end embedded system. We then propose a mitigation technique that uses the hardware-in-the-loop to determine the latency distribution of running the policy on the target platform (onboard compute on aerial robot). A randomly sampled latency from the latency distribution is then added as an artificial delay within the training loop. Training the policy with artificial delays allows us to minimize the hardware gap (discrepancy in the flight time metric reduced from 37.73% to 0.5%). Thus, Air Learning with hardware-in-the-loop characterizes those differences and exposes how the onboard compute’s choice affects the aerial robot’s performance. We also conduct reliability studies to assess the effect of sensor failures on the learned policies. All put together, Air Learning enables a broad class of deep RL research on UAVs. The source code is available at: https://github.com/harvard-edge/AirLearning .
Leveraging Bring Your Own Device for Mobility of Library Reference Services: The Nigerian Perspective

Libraries in Nigeria are increasingly affected by alternative sources of information that provide stiff competition to library reference services. Reference librarians need to adopt a bring your own device (BYOD) strategy to remain relevant. The article explores the practical application of BYOD principles and strategies to the mobility of library reference services in Nigeria. It explores mobile phones’ role in Nigerian libraries, identifies likely challenges in implementing BYOD strategies, and makes recommendations. Reference librarians in Nigeria can adopt BYOD strategies to guarantee the survival and continuous growth of libraries.
Time Series Forecasting using Facebook Prophet for Cloud Resource Management

The heterogeneous nature of workloads running in cloud environments makes future resource usage prediction a complicated problem. Virtual machines can be described in five types of resource utilization patterns: steady, trending, seasonal, cyclic, and bursty behavior. Understanding these usage patterns and behaviors can enhance resource management on cloud data centers, especially VM scheduling, power management, and server health management systems. This paper applies the Facebook Prophet forecast framework on Microsoft Azure VM workload to predict future resource utilization required by the running tasks. We conclude that utilizing data preprocessing and transformation on real virtual machine traces, and incorporating an automatic model hyperparameter tuning process, can significantly increase forecasting accuracy with an average percentage change of over 85%. Furthermore, cloud providers can learn from their data center workloads and employ various forecasting models to gain substantial improvements in cost-efficient resource management.
One-shot action recognition in challenging therapy scenarios

One-shot action recognition aims to recognize new action categories from a single reference example, typically referred to as the anchor example. This work presents a novel approach for one-shot action recognition in the wild that computes motion representations robust to variable kinematic conditions. One-shot action recognition is then performed by evaluating anchor and target motion representations. We also develop a set of complementary steps that boost the action recognition performance in the most challenging scenarios. Our approach is evaluated on the public NTU-120 one-shot action recognition benchmark, outperforming previous action recognition models. Besides, we evaluate our framework on a real use-case of therapy with autistic people. These recordings are particularly challenging due to high-level artifacts from the patient motion. Our results provide not only quantitative but also online qualitative measures, essential for the patient evaluation and monitoring during the actual therapy.
Improved Hybrid Reasoning Approach to Safety Risk Perception under Uncertainty for Mountain Tunnel Construction

Tunnel projects in mountainous areas are considered a kind of high-risk work due to the complexity of geological conditions, as well as the diversity of construction and management factors. Numerous research efforts have been made to estimate the risk level of tunnel collapse by multicriteria decision-making (MCDM) methods considering a variety of risk factors, aiming to provide a decision basis for risk management. However, the uncertainty of fuzziness and randomness existing in evaluation factors gradation are usually neglected. The information fusion method based on extension theory (ET) is an effective tool for uncertain information reasoning. Nevertheless, the potential contribution of each piece of evidence to the fusion result has not been given enough attention. To overcome these deficiencies, this paper presents a novel hybrid reasoning approach to perceive the risk level of mountain tunnel collapse under uncertainty by integrating the extended cloud model (ECM) and improved Dempster-Shafer (D-S) evidence theory at the construction stage. ET and cloud model (CM) are merged to construct the basic probability assignments (BPAs) of the influenceable variables corresponding to different risk states under uncertainty, i.e., the randomness and fuzziness. The overall risk level is derived from a fusion of all the influenceable factors using the improved D-S theory developed in this paper. Specifically, the weight of each piece of evidence is assessed considering its importance to risk perception results and used to improve the evidence source by the discounting method before evidence fusion. Evidence conflicts are resolved through a weighted-average fusion process based on the evidence distance. Besides, a confidence indicator is introduced to assess the reliability of the fusion results. Finally, a case study in Canglongxia Tunnel is employed to verify the validity of the developed approach. The method proposed in this paper has the potential to reduce the deviation of risk assessment results for tunnel collapse in mountain tunnel construction induced by the uncertainty of risk factors.
A lightweight verifiable trust based data collection approach for sensor–cloud systems

A Lightweight Verifiable Trust based Data Collection (LVT-DC) approach is proposed to obtain credible data for mobile vehicles network, in which, a large number of IoT devices are deployed in smart city, and Mobile Vehicles (MVs) collect data from IoT devices and report data to cloud to construct various applications. First, some IoT devices are selected as core-IoT device whose data more than others. Then, Unmanned Aerial Vehicles (UAV) delivers a short verification code to each core-IoT devices and collects its data, and it embeds verification code into the data when the MVs collect it in order for cloud to check. Last, a verifiable trust inference method is proposed which includes two types of trust calculations. (a) Direct trust: MVs If the reported data cannot recover the verification code or is inconsistent with the UAV collection result, reduce the trust of the MVs, (b) Trust inference: If the data of a non-core-IoT devices reported by a MVs is inconsistent with the trusted MVs, then reduce its trust. After a large number of experimental results, the LVT-DC approach can quickly and accurately identify the credibility of the data collector and ensure credible data collection.