Reviewerly/exHarmony huggingface dataset:
Temporal Segment Neural Networks-Enabled Dynamic Hand-Gesture Recognition for Industrial Cyber-Physical Authentication Systems

In recent years, the data security and privacy protection of human cyber-physical systems (CPSs) in the industrial Internet of Things (IoT) have attracted much research interest. In the meantime, applying human–machine identity authentication mechanism to the CPSs has been considered as a promising solution for secure and privacy-enhancing authentication in industrial IoT. In view of the application limitation of traditional authentication methods, especially during the current COVID-19 virus outbreak, the mainstream face recognition and fingerprint recognition methods are constrained by wearing masks and gloves. Recently, with the flourishing studies of artificial intelligence (AI), many AI-based dynamic gesture recognition methods have been developed for the application of human–computer identity recognition. A temporal segment network (TSN) has been designed with a dual-stream convolutional neural network inside to match the flexibility of gesture transformations. Although it has achieved a success in accuracy, the TSN still has the main shortcomings of the insufficient temporal information fusion and the high cost of optical flow feature extraction. In this article, we propose an enhanced temporal segment network (dubbed as <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$e$</tex-math></inline-formula> -TSN). First, hand skeleton features are used instead of optical flow features. Second, a short-term networks (long short-term memory) is utilized as segmental consensus function to improve the accuracy and reduce the feature extraction cost. Experiments demonstrate that the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$e$</tex-math></inline-formula> -TSN can achieve an accuracy of 91.3% on the Jester dataset. Finally, a human–machine identity verification system is developed based on the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$e$</tex-math></inline-formula> -TSN, which can effectually accomplish the human–machine identity in real time and has a high promotion value.
Forecasting solar irradiance with hybrid classical–quantum models: A comprehensive evaluation of deep learning and quantum-enhanced techniques

Predicting solar irradiance has proven to be a challenging task due to its inherently unpredictable and chaotic characteristics. Although machine learning and deep learning models have demonstrated considerable success in this field, the emergence of quantum computing brings a new layer of potential to this complex problem. This paper delves into these emerging methodologies, focusing specifically on their application in the domain of solar irradiance forecasting. This study investigates the integration of quantum layers into a conventional deep feedforward neural network (FFN) and the development of a fully connected quantum neural network (QNN). A bidirectional long short-term memory (BiLSTM) model is employed as a performance benchmark. Among the various models developed within this study, a few have demonstrated notable performance. For example, an FFN composed of five regular layers and two quantum layers (FFN5L2Q), having a total parameter count of 339, generated an error rate of 6.737%. Conversely, another FFN model, featuring eight regular layers and one quantum layer (FFN8L1Q) and having a total parameter count of 5551, exhibited superior performance with an error rate of 4.254%. When compared to the BiLSTM model, which has 49,666 parameters and an error rate of 3.875%, these quantum-enhanced FFNs display robust and competitive performance. This study, therefore, emphasizes the promising prospects of quantum-integrated techniques in augmenting the precision of solar irradiance prediction models.
An Energy-Efficient Tuning Method for Cloud Servers Combining DVFS and Parameter Optimization

Emerging cloud computing applications place a growing demand on resources, leading to increasingly large data centers with significant energy consumption and carbon emissions. Various research conduct optimization methods to improve the energy efficiency of the server in the cloud data center. However, most existing optimization methods are designed for specific applications, thus making it difficult to handle complex cloud environments. In this paper, we propose a general parameter optimization method called MPOD to improve the energy efficiency of cloud servers in real time. MPOD considers issues in the cloud environment, such as SLA guarantee, user privacy, and dynamic workloads. We introduce energy efficiency curves to DVFS, implementing a low-overhead, fast response, and general frequency optimization strategy. Moreover, we design a workload classification framework and three prediction models based on machine learning algorithms to achieve accurate and adaptive Linux kernel parameters optimization. According to the experiment, MPOD can improve the energy efficiency of the server by an average of 30.5%, 20.1%, 10.8% in BenchSEE, SERT and TPC-H, respectively.
Measuring the novelty of scientific publications: A fastText and local outlier factor approach

Although the novelty of scientific publications has been the subject of previous studies, most have examined the distribution of references in the bibliography, which may not be effective in capturing implied scientific knowledge. We propose an analytical framework for measuring the novelty of scientific publications using a paper's title. At the heart of the framework, fastText is used to construct a vector space model in which papers with similar scientific knowledge are located close to each other, and the local outlier factor is used to measure the novelty of scientific knowledge implied in the papers on a numerical scale. The feasibility and validity of the analytical framework were assessed by comparing the average novelty scores of papers recommended with novelty-related tags in Faculty Opinions to those of papers without such tags. This case study of 15,653 papers published in a biomedical journal confirms that our framework is a useful complementary tool for the continuous assessment of the novelty of scientific publications and can serve as a starting point for developing more general models.
Uncertainty Quantification for Machine Learning Output Assurance using Anomaly-based Dataset Dissimilarity Measures

The use of Machine Learning (ML) models as predictive tools has increased dramatically in recent years. However, data-driven systems (such as ML models) exhibit a degree of uncertainty in their predictions. In other words, they could produce unexpectedly erroneous predictions if the uncertainty stemming from the data, choice of model and model parameters is not taken into account. In this paper, we introduce a novel method for quantifying the uncertainty of the performance levels attained by ML classifiers. In particular, we investigate and characterize the uncertainty of model accuracy when classifying out-of-distribution data that is statistically dissimilar from the data employed during training. A main element of this novel Uncertainty Quantification (UQ) method is a measure of the dissimilarity between two datasets. We introduce an innovative family of data dissimilarity measures based on anomaly detection algorithms, namely the Anomaly-based Dataset Dissimilarity (ADD) measures. These dissimilarity measures process feature representations that are derived from the activation values of neural networks when supplied with dataset items. The proposed UQ method for classification performance employs these dissimilarity measures to estimate the classifier accuracy for out-of-distribution datasets, and to give an uncertainty band for those estimates. A numerical analysis of the efficacy of the UQ method is conducted. The obtained results generally demonstrate that the amplitude of the uncertainty band tends to increase as the data dissimilarity measure increases. Overall, this research contributes to the verification and run-time performance prediction of systems composed of ML-based elements.
Reclaiming the Digital Commons: A Public Data Trust for Training Data

Democratization of AI means not only that people can freely use AI, but also that people can collectively decide how AI is to be used. In particular, collective decision-making power is required to redress the negative externalities from the development of increasingly advanced AI systems, including degradation of the digital commons and unemployment from automation. The rapid pace of AI development and deployment currently leaves little room for this power. Monopolized in the hands of private corporations, the development of the most capable foundation models has proceeded largely without public input. There is currently no implemented mechanism for ensuring that the economic value generated by such models is redistributed to account for their negative externalities. The citizens that have generated the data necessary to train models do not have input on how their data are to be used. In this work, we propose that a public data trust assert control over training data for foundation models. In particular, this trust should scrape the internet as a digital commons, to license to commercial model developers for a percentage cut of revenues from deployment. First, we argue in detail for the existence of such a trust. We also discuss feasibility and potential risks. Second, we detail a number of ways for a data trust to incentivize model developers to use training data only from the trust. We propose a mix of verification mechanisms, potential regulatory action, and positive incentives. We conclude by highlighting other potential benefits of our proposed data trust and connecting our work to ongoing efforts in data and compute governance.
Effective Enforceability of EU Competition Law Under AI Development Scenarios

This paper examines whether competition law enforcement can remain effective under different AI development scenarios over the coming years. Economic and political power has become increasingly concentrated into a few AI companies, such as Big Tech. The growth of generative AI could further reinforce this concentration of power in Big Tech. The market power of these companies, and increasingly their involvement in AI, is a major focus for regulators such as the European Commission. Recent EU antitrust fines on Google alone run in the billions. The dynamism of technology markets such as AI can make it difficult for regulators to take effective action. If AI continues to develop rapidly over the coming years, propelled by the proliferation of generative AI, this ability to effectively enforce antitrust law may be further challenged. To help ensure regulators remain effective, EU competition law has been bolstered by a new tech-tailored, ex ante competition regime. These are likely to be critical tools to shape the market power of Big Tech but are largely untested. Exploring how these regulatory tools can be most effective in governing future AI development is a timely question for regulators, lawyers, companies, and citizens. This paper examines this question by considering the ‘effective enforceability’ of EU competition law and the Digital Markets Act under different AI development scenarios. By ‘effective enforceability’ of EU competition law we mean how well it achieves its policy objectives. We consider four factors: jurisdictional scope, potential loopholes, effectiveness of detection, and ability to remedy/sanction breaches. However, there is significant uncertainty as to how AI will develop in the coming years. Considering this, we propose an analytical framework based on five variables: key inputs, speed of development, AI capability, number of actors, and the nature/relationship of actors. In some of these scenarios, we argue EU competition law would struggle to address the power of the largest AI companies; but in many other scenarios it remains a powerful tool. This is a critical juncture for competition regulators. They stand at the dawn of emerging challenges presented by generative AI. With this paper, we hope to contribute to anticipatory governance at this important intersection of legal governance and technology.
Self-Destructing Models: Increasing the Costs of Harmful Dual Uses of Foundation Models

A growing ecosystem of large, open-source foundation models has reduced the labeled data and technical expertise necessary to apply machine learning to many new problems. Yet foundation models pose a clear dual-use risk, indiscriminately reducing the costs of building both harmful and beneficial machine learning systems. Policy tools such as restricted model access and export controls are the primary methods currently used to mitigate such dual-use risks. In this work, we review potential safe-release strategies and argue that both policymakers and AI researchers would benefit from fundamentally new technologies enabling more precise control over the downstream usage of open-source foundation models. We propose one such approach: the task blocking paradigm, in which foundation models are trained with an additional mechanism to impede adaptation to harmful tasks without sacrificing performance on desirable tasks. We call the resulting models self-destructing models, inspired by mechanisms that prevent adversaries from using tools for harmful purposes. We present an algorithm for training self-destructing models leveraging techniques from meta-learning and adversarial learning, which we call meta-learned adversarial censoring (MLAC). In a small-scale experiment, we show MLAC can largely prevent a BERT-style model from being re-purposed to perform gender identification without harming the model's ability to perform profession classification.
Keep Sensors in Check: Disentangling Country-Level Generalization Issues in Mobile Sensor-Based Models with Diversity Scores

Machine learning models trained with passive sensor data from mobile devices can be used to perform various inferences pertaining to activity recognition, context awareness, and health and well-being. Prior work has improved inference performance through the use of multimodal sensors (inertial, GPS, proximity, app usage, etc.) or improved machine learning. In this context, a few studies shed light on critical issues relating to the poor cross-country generalization of models due to distributional shifts across countries. However, these studies have largely relied on inference performance as a means of studying generalization issues, failing to investigate whether the root cause of the problem is linked to specific sensor modalities (independent variables) or the target attribute (dependent variable). In this paper, we study this issue in complex activities of daily living (ADL) inference task, involving 12 classes, by using a multimodal, multi-country dataset collected from 689 participants across eight countries. We first show that the ‘country of origin’ of data is captured by sensors and can be inferred from each modality separately, with an average accuracy of 65%. We then propose two diversity scores (DS) that measure how a country differentiates from others w.r.t. sensor modalities or activities. Using these diversity scores, we observed that both individual sensor modalities and activities have the ability to differentiate countries. However, while many activities capture country differences, only the ‘App usage’ and ‘Location’ sensors can do so. By dissecting country-level diversity across dependent and independent variables, we provide a framework to better understand model generalization issues across countries and country-level diversity of sensing modalities.
Improving Unsupervised Domain Adaptation with Auxiliary Classifier GANs

In this paper, we develop a generative-adversarial-based unsupervised domain adaptation framework to train the classification network and produce source-like images from learned embeddings of both source and target domains simultaneously.1 It is obtained by establishing an interconnection between learned features and an Auxiliary Classifier GAN (ACGAN). To produce source-like images, a deep learning model reconstructs images similar to those in the source dataset. We validated the effectiveness and generality of our method by performing experiments on four adaptation scenarios: USPS to MNIST, MNIST to USPS, SVHN to MNIST, and MNIST to MNISTM. The proposed method works well across four adaptation scenarios.
Self-assessments, attitudes, and motivational orientations towards the use of digital media in teaching a comparison between student teachers of different subject clusters

Findings from research on the education of teachers show that student teachers lack competence in the utilization of digital media and often fail to see the added value in using digital concepts in classrooms. Teacher training institutions are faced with the central challenge of providing student teachers and teachers with adaptive learning opportunities for their competence in digital areas. In the context of various teaching subjects, this raises questions about specific starting points and the actual needs of students, as well as university course offerings. The present study tackles this topic and explores the commonalities and differences between student teachers from three subject clusters: science, technology, engineering, and mathematics (STEM); language literature arts (LLA); and social sciences (SOCS). The questionnaire-based study examines student teachers' self-assessments of their abilities in terms of digital media, attitudes, and motivational orientations towards the use of digital media in teaching. Group comparisons show that STEM student teachers rate themselves better in competence facets such as technological knowledge, technological content knowledge, and technological pedagogical knowledge, although their attitudes and motivation towards learning with digital media do not differ. Despite the different interrelation effects of investigated influencing factors on motivation depending on subject clusters, the findings suggest the promotion of technological competencies and positive attitudes to increase motivation.
Designing an Assistance Tool for Analyzing and Modeling Trainer Activity in Professional Training Through Simulation

Human audience analysis is crucial in numerous domains to understand people's behavior based on their knowledge and environment. In this paper we focus on simulation-based training which has become a popular teaching approach that requires a trainer able to manage a lot of data at the same time. We present tools that are currently being developed to help trainers from two different fields: training for practical teaching gestures and training in civil defense. In this sense, three technological blocks are built to collect and analyze data about trainers' and learners' gestures, gaze, speech and movements. The paper also discusses the future work planned for this project, including the integration of the framework into the Noldus system and its use in civil security training. Overall, the article highlights the potential of technology to improve simulation-based training and provides a roadmap for future development.
Empirical Thresholding on Spatio-Temporal Autoencoders Trained on Surveillance Videos in a Dementia Care Unit

Agitation is a key behavioural and psychological symptom exhibited by people with dementia. These behaviours can put the patient with dementia and others' health and safety at risk. Surveillance cameras installed in long-term care facilities provide an opportunity to monitor patients continuously and flag behaviours of risks, including agitation. However, agitation behaviours occur rarely and diversely, leading to very small training data. Therefore, an anomaly detection approach is more suitable for this problem. In this paper, we train three baseline spatio-temporal convolutional autoencoders (on raw video, skeletons and segmentation mask) on 21 hours of normal activities and tested it on 9 hours of labelled normal and agitation data collected from a real patient in a dementia unit. The deployment of anomaly detection-based classifiers is challenging in real-world due to the absence of a validation set to obtain an operating threshold to regulate true positive and false positive rates. We present a new approach to create a proxy validation set for unseen agitation events utilizing the outliers within normal activities, and trained two separate autoencoders on normal and outliers activities. Then, we present 11 empirical thresholding approaches (existing, adapted and new) using either only normal training data or the proxy validation set. Our results showed consistently across raw video, skeletons and segmentation masks input that incorporating a proxy validation set improved performance both in terms of geometric means and Matthew's correlation coefficient. This paper highlights the real-world deployment challenges and assessment of the limit of true positives or false positives that can be acceptable in a clinical care environment.
InterTrack: Interaction Transformer for 3D Multi-Object Tracking

3D multi-object tracking (MOT) is a key problem for autonomous vehicles, required to perform well-informed motion planning in dynamic environments. Particularly for densely occupied scenes, associating tracks to new detections remains challenging as existing systems tend to omit critical contextual information. Our proposed solution, InterTrack, introduces the Interaction Transformer for 3D MOT to generate discriminative object representations for data association. We extract state and shape features for each track and detection, and efficiently aggregate global information via attention. We then perform a learned regression on each track/detection feature pair to estimate affinities, and use a robust two-stage data association and track management approach to produce the final tracks. We validate our approach on the nuScenes 3D MOT benchmark, where we observe significant improvements, particularly on classes with small physical sizes and clustered objects. As of submission, InterTrack ranks 1 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">st</sup> in overall AMOTA among methods using CenterPoint [1] detections.
Spotting Deep Neural Network Vulnerabilities in Mobile Traffic Forecasting with an Explainable AI Lens

The ability to forecast mobile traffic patterns is key to resource management for mobile network operators and planning for local authorities. Several Deep Neural Networks (DNN) have been designed to capture the complex spatio-temporal characteristics of mobile traffic patterns at scale. These models are complex black boxes whose decisions are inherently hard to explain. Even worse, they have proven vulnerable to adversarial attacks which undermine their applicability in production networks. In this paper, we conduct a first in-depth study of the vulnerabilities of DNNs for large-scale mobile traffic forecasting. We propose DeExp, a new tool that leverages EXplainable Artificial Intelligence (XAI) to understand which Base Stations (BSs) are more influential for forecasting from a spatio-temporal perspective. This is challenging as existing XAI techniques are usually applied to computer vision or natural language processing and need to be adapted to the mobile network context. Upon identifying the more influential BSs, we run state-of-the art Adversarial Machine Learning (AML) techniques on those BSs and measure the accuracy degradation of the predictors. Extensive evaluations with real-world mobile traffic traces pinpoint that attacking BSs relevant to the predictor significantly degrades its accuracy across all the scenarios.