OpenRaiser/Intern-Atlas huggingface dataset:
Bag of States: A Non-sequential Approach to Video-based Engagement Measurement
Venue: arXiv cs.HC 2023
Year: 2023

Automatic measurement of student engagement provides helpful information for instructors to meet learning program objectives and individualize program delivery. Students' behavioral and emotional states need to be analyzed at fine-grained time scales in order to measure their level of engagement. Many existing approaches have developed sequential and spatiotemporal models, such as recurrent neural networks, temporal convolutional networks, and three-dimensional convolutional neural networks, for measuring student engagement from videos. These models are trained to incorporate the order of behavioral and emotional states of students into video analysis and output their level of engagement. In this paper, backed by educational psychology, we question the necessity of modeling the order of behavioral and emotional states of students in measuring their engagement. We develop bag-of-words-based models in which only the occurrence of behavioral and emotional states of students is modeled and analyzed and not the order in which they occur. Behavioral and affective features are extracted from videos and analyzed by the proposed models to determine the level of engagement in an ordinal-output classification setting. Compared to the existing sequential and spatiotemporal approaches for engagement measurement, the proposed non-sequential approach improves the state-of-the-art results. According to experimental results, our method significantly improved engagement level classification accuracy on the IIITB Online SE dataset by 26% compared to sequential models and achieved engagement level classification accuracy as high as 66.58% on the DAiSEE student engagement dataset.
Face Inverse Rendering via Hierarchical Decoupling
Venue: arXiv cs.CV 2023
Year: 2023

Previous face inverse rendering methods often require synthetic data with ground truth and/or professional equipment like a lighting stage. However, a model trained on synthetic data or using pre-defined lighting priors is typically unable to generalize well for real-world situations, due to the gap between synthetic data/lighting priors and real data. Furthermore, for common users, the professional equipment and skill make the task expensive and complex. In this paper, we propose a deep learning framework to disentangle face images in the wild into their corresponding albedo, normal, and lighting components. Specifically, a decomposition network is built with a hierarchical subdivision strategy, which takes image pairs captured from arbitrary viewpoints as input. In this way, our approach can greatly mitigate the pressure from data preparation, and significantly broaden the applicability of face inverse rendering. Extensive experiments are conducted to demonstrate the efficacy of our design, and show its superior performance in face relighting over other state-of-the-art alternatives. {Our code is available at \url{https://github.com/AutoHDR/HD-Net.git}}
Subgraph Centralization: A Necessary Step for Graph Anomaly Detection
Venue: arXiv cs.LG 2023
Year: 2023

Graph anomaly detection has attracted a lot of interest recently. Despite their successes, existing detectors have at least two of the three weaknesses: (a) high computational cost which limits them to small-scale networks only; (b) existing treatment of subgraphs produces suboptimal detection accuracy; and (c) unable to provide an explanation as to why a node is anomalous, once it is identified. We identify that the root cause of these weaknesses is a lack of a proper treatment for subgraphs. A treatment called Subgraph Centralization for graph anomaly detection is proposed to address all the above weaknesses. Its importance is shown in two ways. First, we present a simple yet effective new framework called Graph-Centric Anomaly Detection (GCAD). The key advantages of GCAD over existing detectors including deep-learning detectors are: (i) better anomaly detection accuracy; (ii) linear time complexity with respect to the number of nodes; and (iii) it is a generic framework that admits an existing point anomaly detector to be used to detect node anomalies in a network. Second, we show that Subgraph Centralization can be incorporated into two existing detectors to overcome the above-mentioned weaknesses.
A Combined Finite Element and Finite Volume Method for Liquid Simulation
Venue: arXiv cs.GR 2023
Year: 2023

We introduce a new Eulerian simulation framework for liquid animation that leverages both finite element and finite volume methods. In contrast to previous methods where the whole simulation domain is discretized either using the finite volume method or finite element method, our method spatially merges them together using two types of discretization being tightly coupled on its seams while enforcing second order accurate boundary conditions at free surfaces. We achieve our formulation via a variational form using new shape functions specifically designed for this purpose. By enabling a mixture of the two methods, we can take advantage of the best of two worlds. For example, finite volume method (FVM) result in sparse linear systems; however, complexity is encountered when unstructured grids such as tetrahedral or Voronoi elements are used. Finite element method (FEM), on the other hand, result in comparably denser linear systems, but the complexity remains the same even if unstructured elements are chosen; thereby facilitating spatial adaptivity. In this paper, we propose to use FVM for the majority parts to retain the sparsity of linear systems and FEM for parts where the grid elements are allowed to be freely deformed. An example of this application is locally moving grids. We show that by adapting the local grid movement to an underlying nearly rigid motion, numerical diffusion is noticeably reduced; leading to better preservation of structural details such as sharp edges, thin sheets and spindles of liquids.
Design, Modeling and Control of a Top-loading Fully-Actuated Cargo Transportation Multirotor
Venue: arXiv cs.RO 2023
Year: 2023

Existing multirotor-based cargo transportation does not maintain a constant cargo attitude due to underactuation; however, fragile payloads may require a consistent posture. The conventional method is also cumbersome when loading cargo, and the size of the cargo to be loaded is limited. To overcome these issues, we propose a new fully-actuated multirotor unmanned aerial vehicle platform capable of translational motion while maintaining a constant attitude. Our newly developed platform has a cubic exterior and can freely place cargo at any point on the flat top surface. However, the center-of-mass (CoM) position changes when cargo is loaded, leading to undesired attitudinal motion due to unwanted torque generation. To address this problem, we introduce a new model-free center-of-mass position estimation method inspired by the extremum-seeking control (ESC) technique. Experimental results are presented to validate the performance of the proposed estimation method, effectively estimating the CoM position and showing satisfactory constant-attitude flight performance.
Graph-based Keyword Planning for Legal Clause Generation from Topics
Venue: arXiv cs.CL 2023
Year: 2023

Generating domain-specific content such as legal clauses based on minimal user-provided information can be of significant benefit in automating legal contract generation. In this paper, we propose a controllable graph-based mechanism that can generate legal clauses using only the topic or type of the legal clauses. Our pipeline consists of two stages involving a graph-based planner followed by a clause generator. The planner outlines the content of a legal clause as a sequence of keywords in the order of generic to more specific clause information based on the input topic using a controllable graph-based mechanism. The generation stage takes in a given plan and generates a clause. The pipeline consists of a graph-based planner followed by text generation. We illustrate the effectiveness of our proposed two-stage approach on a broad set of clause topics in contracts.
Blockchain based Resource Governance for Decentralized Web Environments
Venue: arXiv cs.NI 2023
Year: 2023

Decentralization initiatives such as Solid and ActivityPub aim to give data owners more control over their data and to level the playing field by enabling small companies and individuals to gain access to data, thus stimulating innovation. However, these initiatives typically employ access control mechanisms that cannot verify compliance with usage conditions after access has been granted to others. In this paper, we extend the state of the art by proposing a resource governance conceptual framework, entitled ReGov, that facilitates usage control in decentralized web environments. We subsequently demonstrate how our framework can be instantiated by combining blockchain and trusted execution environments. Through blockchain technologies, we record policies expressing the usage conditions associated with resources and monitor their compliance. Our instantiation employs trusted execution environments to enforce said policies, inside data consumers' devices.} We evaluate the framework instantiation through a detailed analysis of requirements derived from a data market motivating scenario, as well as an assessment of the security, privacy, and affordability aspects of our proposal.
FewSOME: One-Class Few Shot Anomaly Detection with Siamese Networks
Venue: arXiv cs.LG 2023
Year: 2023

Recent Anomaly Detection techniques have progressed the field considerably but at the cost of increasingly complex training pipelines. Such techniques require large amounts of training data, resulting in computationally expensive algorithms that are unsuitable for settings where only a small amount of normal samples are available for training. We propose 'Few Shot anOMaly detection' (FewSOME), a deep One-Class Anomaly Detection algorithm with the ability to accurately detect anomalies having trained on 'few' examples of the normal class and no examples of the anomalous class. We describe FewSOME to be of low complexity given its low data requirement and short training time. FewSOME is aided by pretrained weights with an architecture based on Siamese Networks. By means of an ablation study, we demonstrate how our proposed loss, 'Stop Loss', improves the robustness of FewSOME. Our experiments demonstrate that FewSOME performs at state-of-the-art level on benchmark datasets MNIST, CIFAR-10, F-MNIST and MVTec AD while training on only 30 normal samples, a minute fraction of the data that existing methods are trained on. Moreover, our experiments show FewSOME to be robust to contaminated datasets. We also report F1 score and balanced accuracy in addition to AUC as a benchmark for future techniques to be compared against. Code available; https://github.com/niamhbelton/FewSOME.
SECOMlint: A linter for Security Commit Messages
Venue: arXiv cs.CR 2023
Year: 2023

Transparent and efficient vulnerability and patch disclosure are still a challenge in the security community, essentially because of the poor-quality documentation stemming from the lack of standards. SECOM is a recently-proposed standard convention for security commit messages that enables the writing of well-structured and complete commit messages for security patches. The convention prescribes different bits of security-related information essential for a better understanding of vulnerabilities by humans and tools. SECOMlint is an automated and configurable solution to help security and maintenance teams infer compliance against the SECOM standard when submitting patches to security vulnerabilities in their source version control systems. The tool leverages the natural language processing technique Named-Entity Recognition (NER) to extract security-related information from commit messages and uses it to match the compliance standards designed. We demonstrate SECOMlint at https://youtu.be/-1hzpMN_uFI; and documentation and its source code at https://tqrg.github.io/secomlint/.
Byzantine Resilience at Swarm Scale: A Decentralized Blocklist Protocol from Inter-robot Accusations
Venue: arXiv cs.RO 2023
Year: 2023

The Weighted-Mean Subsequence Reduced (W-MSR) algorithm, the state-of-the-art method for Byzantine-resilient design of decentralized multi-robot systems, is based on discarding outliers received over Linear Consensus Protocol (LCP). Although W-MSR provides well-understood theoretical guarantees relating robust network connectivity to the convergence of the underlying consensus, the method comes with several limitations preventing its use at scale: (1) the number of Byzantine robots, F, to tolerate should be known a priori, (2) the requirement that each robot maintains 2F+1 neighbors is impractical for large F, (3) information propagation is hindered by the requirement that F+1 robots independently make local measurements of the consensus property in order for the swarm's decision to change, and (4) W-MSR is specific to LCP and does not generalize to applications not implemented over LCP. In this work, we propose a Decentralized Blocklist Protocol (DBP) based on inter-robot accusations. Accusations are made on the basis of locally-made observations of misbehavior, and once shared by cooperative robots across the network are used as input to a graph matching algorithm that computes a blocklist. DBP generalizes to applications not implemented via LCP, is adaptive to the number of Byzantine robots, and allows for fast information propagation through the multi-robot system while simultaneously reducing the required network connectivity relative to W-MSR. On LCP-type applications, DBP reduces the worst-case connectivity requirement of W-MSR from (2F+1)-connected to (F+1)-connected and the number of cooperative observers required to propagate new information from F+1 to just 1 observer. We demonstrate empirically that our approach to Byzantine resilience scales to hundreds of robots on cooperative target tracking, time synchronization, and localization case studies.
Adaptive Deep Neural Network Inference Optimization with EENet
Venue: arXiv cs.LG 2023
Year: 2023

Well-trained deep neural networks (DNNs) treat all test samples equally during prediction. Adaptive DNN inference with early exiting leverages the observation that some test examples can be easier to predict than others. This paper presents EENet, a novel early-exiting scheduling framework for multi-exit DNN models. Instead of having every sample go through all DNN layers during prediction, EENet learns an early exit scheduler, which can intelligently terminate the inference earlier for certain predictions, which the model has high confidence of early exit. As opposed to previous early-exiting solutions with heuristics-based methods, our EENet framework optimizes an early-exiting policy to maximize model accuracy while satisfying the given per-sample average inference budget. Extensive experiments are conducted on four computer vision datasets (CIFAR-10, CIFAR-100, ImageNet, Cityscapes) and two NLP datasets (SST-2, AgNews). The results demonstrate that the adaptive inference by EENet can outperform the representative existing early exit techniques. We also perform a detailed visualization analysis of the comparison results to interpret the benefits of EENet.
Genetic Imitation Learning by Reward Extrapolation
Venue: arXiv cs.NE 2023
Year: 2023

Imitation learning demonstrates remarkable performance in various domains. However, imitation learning is also constrained by many prerequisites. The research community has done intensive research to alleviate these constraints, such as adding the stochastic policy to avoid unseen states, eliminating the need for action labels, and learning from the suboptimal demonstrations. Inspired by the natural reproduction process, we proposed a method called GenIL that integrates the Genetic Algorithm with imitation learning. The involvement of the Genetic Algorithm improves the data efficiency by reproducing trajectories with various returns and assists the model in estimating more accurate and compact reward function parameters. We tested GenIL in both Atari and Mujoco domains, and the result shows that it successfully outperforms the previous extrapolation methods over extrapolation accuracy, robustness, and overall policy performance when input data is limited.
Chip Guard ECC: An Efficient, Low Latency Method
Venue: arXiv cs.AR 2023
Year: 2023

Chip Guard is a new approach to symbol-correcting error correction codes. It can be scaled to various data burst sizes and reliability levels. A specific version for DDR5 is described. It uses the usual DDR5 configuration of 8 data chips, plus 2 chips for ECC and metadata, with 64-bit bursts per chip, to support whole-chip correction reliably and with high probity (reporting of uncorrectable faults). Various numbers of metadata bits may be supported with defined tradeoffs for reliability and probity. The method should correct all bounded faults of a single chip, with less than 1 in 10^12 chance of failing to correct unbounded faults in one chip, or less than 1 in 10^12 chance of failure to detect an uncorrected fault which affects multiple chips.
Improve Noise Tolerance of Robust Loss via Noise-Awareness
Venue: arXiv cs.LG 2023
Year: 2023

Robust loss minimization is an important strategy for handling robust learning issue on noisy labels. Current approaches for designing robust losses involve the introduction of noise-robust factors, i.e., hyperparameters, to control the trade-off between noise robustness and learnability. However, finding suitable hyperparameters for different datasets with noisy labels is a challenging and time-consuming task. Moreover, existing robust loss methods usually assume that all training samples share common hyperparameters, which are independent of instances. This limits the ability of these methods to distinguish the individual noise properties of different samples and overlooks the varying contributions of diverse training samples in helping models understand underlying patterns. To address above issues, we propose to assemble robust loss with instance-dependent hyperparameters to improve their noise tolerance with theoretical guarantee. To achieve setting such instance-dependent hyperparameters for robust loss, we propose a meta-learning method which is capable of adaptively learning a hyperparameter prediction function, called Noise-Aware-Robust-Loss-Adjuster (NARL-Adjuster for brevity). Through mutual amelioration between hyperparameter prediction function and classifier parameters in our method, both of them can be simultaneously finely ameliorated and coordinated to attain solutions with good generalization capability. Four SOTA robust loss functions are attempted to be integrated with our algorithm, and comprehensive experiments substantiate the general availability and effectiveness of the proposed method in both its noise tolerance and performance.
One Size Does not Fit All: Quantifying the Risk of Malicious App Encounters for Different Android User Profiles
Venue: arXiv cs.CR 2023
Year: 2023

Previous work has investigated the particularities of security practices within specific user communities defined based on country of origin, age, prior tech abuse, and economic status. Their results highlight that current security solutions that adopt a one-size-fits-all-users approach ignore the differences and needs of particular user communities. However, those works focus on a single community or cluster users into hard-to-interpret sub-populations. In this work, we perform a large-scale quantitative analysis of the risk of encountering malware and other potentially unwanted applications (PUA) across user communities. At the core of our study is a dataset of app installation logs collected from 12M Android mobile devices. Leveraging user-installed apps, we define intuitive profiles based on users' interests (e.g., gamers and investors), and fit a subset of 5.4M devices to those profiles. Our analysis is structured in three parts. First, we perform risk analysis on the whole population to measure how the risk of malicious app encounters is affected by different factors. Next, we create different profiles to investigate whether risk differences across users may be due to their interests. Finally, we compare a per-profile approach for classifying clean and infected devices with the classical approach that considers the whole population. We observe that features such as the diversity of the app signers and the use of alternative markets highly correlate with the risk of malicious app encounters. We also discover that some profiles such as gamers and social-media users are exposed to more than twice the risks experienced by the average users. We also show that the classification outcome has a marked accuracy improvement when using a per-profile approach to train the prediction models. Overall, our results confirm the inadequacy of one-size-fits-all protection solutions.
GeoCovaxTweets: COVID-19 Vaccines and Vaccination-specific Global Geotagged Twitter Conversations
Venue: arXiv cs.SI 2023
Year: 2023

Social media platforms provide actionable information during crises and pandemic outbreaks. The COVID-19 pandemic has imposed a chronic public health crisis worldwide, with experts considering vaccines as the ultimate prevention to achieve herd immunity against the virus. A proportion of people may turn to social media platforms to oppose vaccines and vaccination, hindering government efforts to eradicate the virus. This paper presents the COVID-19 vaccines and vaccination-specific global geotagged tweets dataset, GeoCovaxTweets, that contains more than 1.8 million tweets, with location information and longer temporal coverage, originating from 233 countries and territories between January 2020 and November 2022. The paper discusses the dataset's curation method and how it can be re-created locally, and later explores the dataset through multiple tweets distributions and briefly discusses its potential use cases. We anticipate that the dataset will assist the researchers in the crisis computing domain to explore the conversational dynamics of COVID-19 vaccines and vaccination Twitter discourse through numerous spatial and temporal dimensions concerning trends, shifts in opinions, misinformation, and anti-vaccination campaigns.
Compression of GPS Trajectories using Autoencoders
Venue: arXiv cs.LG 2023
Year: 2023

The ubiquitous availability of mobile devices capable of location tracking led to a significant rise in the collection of GPS data. Several compression methods have been developed in order to reduce the amount of storage needed while keeping the important information. In this paper, we present an lstm-autoencoder based approach in order to compress and reconstruct GPS trajectories, which is evaluated on both a gaming and real-world dataset. We consider various compression ratios and trajectory lengths. The performance is compared to other trajectory compression algorithms, i.e., Douglas-Peucker. Overall, the results indicate that our approach outperforms Douglas-Peucker significantly in terms of the discrete Fréchet distance and dynamic time warping. Furthermore, by reconstructing every point lossy, the proposed methodology offers multiple advantages over traditional methods.