OpenRaiser/Intern-Atlas huggingface dataset:
Natural Language Processing for Systems Engineering: Automatic Generation of Systems Modelling Language Diagrams
Venue: arXiv cs.CL 2022
Year: 2022

The design of complex engineering systems is an often long and articulated process that highly relies on engineers' expertise and professional judgment. As such, the typical pitfalls of activities involving the human factor often manifest themselves in terms of lack of completeness or exhaustiveness of the analysis, inconsistencies across design choices or documentation, as well as an implicit degree of subjectivity. An approach is proposed to assist systems engineers in the automatic generation of systems diagrams from unstructured natural language text. Natural Language Processing (NLP) techniques are used to extract entities and their relationships from textual resources (e.g., specifications, manuals, technical reports, maintenance reports) available within an organisation, and convert them into Systems Modelling Language (SysML) diagrams, with particular focus on structure and requirement diagrams. The intention is to provide the users with a more standardised, comprehensive and automated starting point onto which subsequently refine and adapt the diagrams according to their needs. The proposed approach is flexible and open-domain. It consists of six steps which leverage open-access tools, and it leads to an automatic generation of SysML diagrams without intermediate modelling requirement, but through the specification of a set of parameters by the user. The applicability and benefits of the proposed approach are shown through six case studies having different textual sources as inputs, and benchmarked against manually defined diagram elements.
Rapid design space exploration of multi-clock domain MPSoCs with hybrid prototyping
Venue: arXiv cs.DC 2022
Year: 2022

This paper presents novel techniques of using hybrid prototyping for early power-performance analysis of MPSoC designs with multiple clock domains. The fundamental idea of hybrid prototyping is to simulate a design with multiple cores by creating an emulation kernel in software on top of a single physical instance of the core. However, so far hybrid prototyping has been limited to homogeneous multicores running at the same clock frequency. Moreover, hybrid prototyping has not yet been demonstrated for efficient design space exploration. Our work focuses on enhancing the capabilities of hybrid prototyping, such that it can be applied to realistic multi-clock MPSoC designs as well to perform early power-performance evaluation of MPSoC designs. Our experiments using industrial strength applications such as JPEG, MP3 and Packet Processing, demonstrate the high accuracy of our hybrid prototypes, and over two orders of magnitude improvement over software simulation speed. We also demonstrate that exploring over 150 design options using hybrid prototyping can be done with high reliability in the order of minutes compared to multiple days using conventional FPGA prototyping.
Experimenting with Experimentation: Rethinking The Role of Experimentation in Educational Design
Venue: arXiv cs.HC 2022
Year: 2022

What if we take a broader view of what it means to run an education experiment? In this paper, we explore opportunities that arise when we think beyond the commonly-held notion that the purpose of an experiment is to either accept or reject a pre-defined hypothesis and instead, reconsider experimentation as a means to explore the complex design space of creating and improving instructional content. This is an approach we call experiment-inspired design. Then, to operationalize these ideas in a real-world experimentation venue, we investigate the implications of running a sequence of interventions teaching first-year students "meta-skills": transferable skills applicable to multiple areas of their lives, such as planning, and managing stress. Finally, using two examples as case studies for meta-skills interventions (stress-reappraisal and mental contrasting with implementation intentions), we reflect on our experiences with experiment-inspired design and share six preliminary lessons on how to use experimentation for design.
Interpretable Polynomial Neural Ordinary Differential Equations
Venue: arXiv cs.LG 2022
Year: 2022

Neural networks have the ability to serve as universal function approximators, but they are not interpretable and don't generalize well outside of their training region. Both of these issues are problematic when trying to apply standard neural ordinary differential equations (neural ODEs) to dynamical systems. We introduce the polynomial neural ODE, which is a deep polynomial neural network inside of the neural ODE framework. We demonstrate the capability of polynomial neural ODEs to predict outside of the training region, as well as perform direct symbolic regression without additional tools such as SINDy.
PECCO: A Profit and Cost-oriented Computation Offloading Scheme in Edge-Cloud Environment with Improved Moth-flame Optimisation
Venue: arXiv cs.DC 2022
Year: 2022

With the fast growing quantity of data generated by smart devices and the exponential surge of processing demand in the Internet of Things (IoT) era, the resource-rich cloud centres have been utilised to tackle these challenges. To relieve the burden on cloud centres, edge-cloud computation offloading becomes a promising solution since shortening the proximity between the data source and the computation by offloading computation tasks from the cloud to edge devices can improve performance and Quality of Service (QoS). Several optimisation models of edge-cloud computation offloading have been proposed that take computation costs and heterogeneous communication costs into account. However, several important factors are not jointly considered, such as heterogeneities of tasks, load balancing among nodes and the profit yielded by computation tasks, which lead to the profit and cost-oriented computation offloading optimisation model PECCO proposed in this paper. Considering that the model is hard in nature and the optimisation objective is not differentiable, we propose an improved Moth-flame optimiser PECCO-MFI which addresses some deficiencies of the original Moth-flame Optimiser and integrate it under the edge-cloud environment. Comprehensive experiments are conducted to verify the superior performance of the proposed method when optimising the proposed task offloading model under the edge-cloud environment.
Using Adaptive Experiments to Rapidly Help Students
Venue: arXiv cs.LG 2022
Year: 2022

Adaptive experiments can increase the chance that current students obtain better outcomes from a field experiment of an instructional intervention. In such experiments, the probability of assigning students to conditions changes while more data is being collected, so students can be assigned to interventions that are likely to perform better. Digital educational environments lower the barrier to conducting such adaptive experiments, but they are rarely applied in education. One reason might be that researchers have access to few real-world case studies that illustrate the advantages and disadvantages of these experiments in a specific context. We evaluate the effect of homework email reminders in students by conducting an adaptive experiment using the Thompson Sampling algorithm and compare it to a traditional uniform random experiment. We present this as a case study on how to conduct such experiments, and we raise a range of open questions about the conditions under which adaptive randomized experiments may be more or less useful.
Robotics in Snow and Ice
Venue: arXiv cs.RO 2022
Year: 2022

Definition: The terms "robotics in snow and ice" refers to robotic systems being studied, developed, and used in areas where water can be found in its solid state. This specialized branch of field robotics investigates the impact of extreme conditions related to cold environments on autonomous vehicles.
Classifier Transfer with Data Selection Strategies for Online Support Vector Machine Classification with Class Imbalance
Venue: arXiv cs.LG 2022
Year: 2022

Objective: Classifier transfers usually come with dataset shifts. To overcome them, online strategies have to be applied. For practical applications, limitations in the computational resources for the adaptation of batch learning algorithms, like the SVM, have to be considered. Approach: We review and compare several strategies for online learning with SVMs. We focus on data selection strategies which limit the size of the stored training data [...] Main Results: For different data shifts, different criteria are appropriate. For the synthetic data, adding all samples to the pool of considered samples performs often significantly worse than other criteria. Especially, adding only misclassified samples performed astoundingly well. Here, balancing criteria were very important when the other criteria were not well chosen. For the transfer setups, the results show that the best strategy depends on the intensity of the drift during the transfer. Adding all and removing the oldest samples results in the best performance, whereas for smaller drifts, it can be sufficient to only add potential new support vectors of the SVM which reduces processing resources. Significance: For BCIs based on EEG models, trained on data from a calibration session, a previous recording session, or even from a recording session with one or several other subjects, are used. This transfer of the learned model usually decreases the performance and can therefore benefit from online learning which adapts the classifier like the established SVM. We show that by using the right combination of data selection criteria, it is possible to adapt the classifier and largely increase the performance. Furthermore, in some cases it is possible to speed up the processing and save computational by updating with a subset of special samples and keeping a small subset of samples for training the classifier.
Blockchain-based Edge Resource Sharing for Metaverse
Venue: arXiv cs.DC 2022
Year: 2022

Although Metaverse has recently been widely studied, its practical application still faces many challenges. One of the severe challenges is the lack of sufficient resources for computing and communication on local devices, resulting in the inability to access the Metaverse services. To address this issue, this paper proposes a practical blockchain-based mobile edge computing (MEC) platform for resource sharing and optimal utilization to complete the requested offloading tasks, given the heterogeneity of servers' available resources and that of users' task requests. To be specific, we first elaborate the design of our proposed system and then dive into the task allocation mechanism to assign offloading tasks to proper servers. To solve the multiple task allocation (MTA) problem in polynomial time, we devise a learning-based algorithm. Since the objective function and constraints of MTA are significantly affected by the servers uploading the tasks, we reformulate it as a reinforcement learning problem and calculate the rewards for each state and action considering the influences of servers. Finally, numerous experiments are conducted to demonstrate the effectiveness and efficiency of our proposed system and algorithms.
Auto-ViT-Acc: An FPGA-Aware Automatic Acceleration Framework for Vision Transformer with Mixed-Scheme Quantization
Venue: arXiv cs.CV 2022
Year: 2022

Vision transformers (ViTs) are emerging with significantly improved accuracy in computer vision tasks. However, their complex architecture and enormous computation/storage demand impose urgent needs for new hardware accelerator design methodology. This work proposes an FPGA-aware automatic ViT acceleration framework based on the proposed mixed-scheme quantization. To the best of our knowledge, this is the first FPGA-based ViT acceleration framework exploring model quantization. Compared with state-of-the-art ViT quantization work (algorithmic approach only without hardware acceleration), our quantization achieves 0.47% to 1.36% higher Top-1 accuracy under the same bit-width. Compared with the 32-bit floating-point baseline FPGA accelerator, our accelerator achieves around 5.6x improvement on the frame rate (i.e., 56.8 FPS vs. 10.0 FPS) with 0.71% accuracy drop on ImageNet dataset for DeiT-base.
Real-Time Oil Leakage Detection on Aftermarket Motorcycle Damping System with Convolutional Neural Networks
Venue: arXiv cs.CV 2022
Year: 2022

In this work, we describe in detail how Deep Learning and Computer Vision can help to detect fault events of the AirTender system, an aftermarket motorcycle damping system component. One of the most effective ways to monitor the AirTender functioning is to look for oil stains on its surface. Starting from real-time images, AirTender is first detected in the motorbike suspension system, simulated indoor, and then, a binary classifier determines whether AirTender is spilling oil or not. The detection is made with the help of the Yolo5 architecture, whereas the classification is carried out with the help of a suitably designed Convolutional Neural Network, OilNet40. In order to detect oil leaks more clearly, we dilute the oil in AirTender with a fluorescent dye with an excitation wavelength peak of approximately 390 nm. AirTender is then illuminated with suitable UV LEDs. The whole system is an attempt to design a low-cost detection setup. An on-board device, such as a mini-computer, is placed near the suspension system and connected to a full hd camera framing AirTender. The on-board device, through our Neural Network algorithm, is then able to localize and classify AirTender as normally functioning (non-leak image) or anomaly (leak image).
Multi-View Pre-Trained Model for Code Vulnerability Identification
Venue: arXiv cs.SE 2022
Year: 2022

Vulnerability identification is crucial for cyber security in the software-related industry. Early identification methods require significant manual efforts in crafting features or annotating vulnerable code. Although the recent pre-trained models alleviate this issue, they overlook the multiple rich structural information contained in the code itself. In this paper, we propose a novel Multi-View Pre-Trained Model (MV-PTM) that encodes both sequential and multi-type structural information of the source code and uses contrastive learning to enhance code representations. The experiments conducted on two public datasets demonstrate the superiority of MV-PTM. In particular, MV-PTM improves GraphCodeBERT by 3.36\% on average in terms of F1 score.
Multi-scale Feature Aggregation for Crowd Counting
Venue: arXiv cs.CV 2022
Year: 2022

Convolutional Neural Network (CNN) based crowd counting methods have achieved promising results in the past few years. However, the scale variation problem is still a huge challenge for accurate count estimation. In this paper, we propose a multi-scale feature aggregation network (MSFANet) that can alleviate this problem to some extent. Specifically, our approach consists of two feature aggregation modules: the short aggregation (ShortAgg) and the skip aggregation (SkipAgg). The ShortAgg module aggregates the features of the adjacent convolution blocks. Its purpose is to make features with different receptive fields fused gradually from the bottom to the top of the network. The SkipAgg module directly propagates features with small receptive fields to features with much larger receptive fields. Its purpose is to promote the fusion of features with small and large receptive fields. Especially, the SkipAgg module introduces the local self-attention features from the Swin Transformer blocks to incorporate rich spatial information. Furthermore, we present a local-and-global based counting loss by considering the non-uniform crowd distribution. Extensive experiments on four challenging datasets (ShanghaiTech dataset, UCF_CC_50 dataset, UCF-QNRF Dataset, WorldExpo'10 dataset) demonstrate the proposed easy-to-implement MSFANet can achieve promising results when compared with the previous state-of-the-art approaches.
Empirical Formal Methods: Guidelines for Performing Empirical Studies on Formal Methods
Venue: arXiv cs.SE 2022
Year: 2022

Empirical studies on formal methods and tools are rare. In this paper, we provide guidelines for such studies. We mention their main ingredients and then define nine different study strategies (laboratory experiments with software and human subjects, usability testing, surveys, qualitative studies, judgment studies, case studies, systematic literature reviews, and systematic mapping studies) and discuss for each of them their crucial characteristics, the difficulties of applying them to formal methods and tools, typical threats to validity, their maturity in formal methods, pointers to external guidelines, and pointers to studies in other fields. We conclude with a number of challenges for empirical formal methods.
A Novel Resource Allocation for Anti-jamming in Cognitive-UAVs: an Active Inference Approach
Venue: arXiv cs.LG 2022
Year: 2022

This work proposes a novel resource allocation strategy for anti-jamming in Cognitive Radio using Active Inference ($\textit{AIn}$), and a cognitive-UAV is employed as a case study. An Active Generalized Dynamic Bayesian Network (Active-GDBN) is proposed to represent the external environment that jointly encodes the physical signal dynamics and the dynamic interaction between UAV and jammer in the spectrum. We cast the action and planning as a Bayesian inference problem that can be solved by avoiding surprising states (minimizing abnormality) during online learning. Simulation results verify the effectiveness of the proposed $\textit{AIn}$ approach in minimizing abnormalities (maximizing rewards) and has a high convergence speed by comparing it with the conventional Frequency Hopping and Q-learning.
The Completeness of Reasoning Algorithms for Clause Sets in Description Logic ALC
Venue: arXiv cs.LO 2022
Year: 2022

On the Semantic Web, metadata and ontologies are used to enable computers to read data. The Web Ontology Language (OWL) has been proposed as a standard ontological language, and various inference systems for this language have been studied. Description logics are regarded as the theoretical foundations of OWL; they provide the syntax and semantics of a formal language for describing ontologies and knowledge bases. In addition, tableau algorithms for description logics have been developed as the standard reasoning algorithms for decidable problems. However, tableau algorithms generate inefficient reasoning steps owing to their nondeterministic branching for disjunction as well as the increase in the size of models occasioned by existential quantification. In this study, we propose conjunctive normal form (CNF) concepts, which utilize a flat concept form for description logic ALC in order to develop algorithms for reasoning about sets of clauses. We present an efficient reasoning algorithm for clause sets where any ALC concept is transformed into an equivalent CNF concept. Theoretically, we prove the soundness, completeness, and termination of the reasoning algorithms for the satisfiability of CNF concepts.
Explaining Machine Learning DGA Detectors from DNS Traffic Data
Venue: arXiv cs.CR 2022
Year: 2022

One of the most common causes of lack of continuity of online systems stems from a widely popular Cyber Attack known as Distributed Denial of Service (DDoS), in which a network of infected devices (botnet) gets exploited to flood the computational capacity of services through the commands of an attacker. This attack is made by leveraging the Domain Name System (DNS) technology through Domain Generation Algorithms (DGAs), a stealthy connection strategy that yet leaves suspicious data patterns. To detect such threats, advances in their analysis have been made. For the majority, they found Machine Learning (ML) as a solution, which can be highly effective in analyzing and classifying massive amounts of data. Although strongly performing, ML models have a certain degree of obscurity in their decision-making process. To cope with this problem, a branch of ML known as Explainable ML tries to break down the black-box nature of classifiers and make them interpretable and human-readable. This work addresses the problem of Explainable ML in the context of botnet and DGA detection, which at the best of our knowledge, is the first to concretely break down the decisions of ML classifiers when devised for botnet/DGA detection, therefore providing global and local explanations.
Learning to Improve Code Efficiency
Venue: arXiv cs.SE 2022
Year: 2022

Improvements in the performance of computing systems, driven by Moore's Law, have transformed society. As such hardware-driven gains slow down, it becomes even more important for software developers to focus on performance and efficiency during development. While several studies have demonstrated the potential from such improved code efficiency (e.g., 2x better generational improvements compared to hardware), unlocking these gains in practice has been challenging. Reasoning about algorithmic complexity and the interaction of coding patterns on hardware can be challenging for the average programmer, especially when combined with pragmatic constraints around development velocity and multi-person development. This paper seeks to address this problem. We analyze a large competitive programming dataset from the Google Code Jam competition and find that efficient code is indeed rare, with a 2x runtime difference between the median and the 90th percentile of solutions. We propose using machine learning to automatically provide prescriptive feedback in the form of hints, to guide programmers towards writing high-performance code. To automatically learn these hints from the dataset, we propose a novel discrete variational auto-encoder, where each discrete latent variable represents a different learned category of code-edit that increases performance. We show that this method represents the multi-modal space of code efficiency edits better than a sequence-to-sequence baseline and generates a distribution of more efficient solutions.
Using query frequencies in tree-based revocation for certificateless authentication in VANETs
Venue: arXiv cs.CR 2022
Year: 2022

Revocation of dishonest users is not an easy problem. This paper proposes a new way to manage revocation of pseudonyms in vehicular ad-hoc networks when using identity-based authentication to increase efficiency and security through certificateless authentication. In order to improve the performance of revocation lists, this paper proposes the use of a data structure based on authenticated dynamic hash k-ary trees and the frequency with which revoked pseudonyms are consulted. The use of the knowledge about the frequency of consultation of revoked pseudonyms allows an easier access to the most popular revoked pseudonyms to the detriment of revoked pseudonyms that are the least consulted. Accordingly, the proposal is especially useful in urban environments where there are vehicles that spend more time on road than others, such as public service vehicles.