Reviewerly/exHarmony huggingface dataset:
Token Packing for Transformers with Variable-Length Inputs

Transformer-based models has achieved remarkable success in extensive tasks for natural language processing. To face the variable-length sentences in human language, popular deep learning frameworks rely on zero padding for batch processing, which introduces significant computation and memory overhead. Existing works attempt to eliminate padding redundancy but results in low hardware efficiency due to the mismatch between the variable shape of operations and fixed shape of processing elements (PEs). This paper proposes a reconfigurable systolic array with token packing in three folds to boost hardware efficiency. First, matrix multiplications for different tokens can be packed along the array columns to improve spatial efficiency. Meanwhile, for temporal efficiency, we develop a coarse-grained pipeline for attention, where stages can run on different parts of the array at the same time. We further exploit the masking redundancy in the Transformer decoder with runtime reconfigurable inter-PE connection and buffer switching. Applied to GPT, our FPGA design has achieved 1.16× higher normalized throughput and 1.94× better runtime MAC utilization over the state-of-the-art GPU performance for variable-length input sequences from GLUE and SQuAD dataset.
Co-ViSu: a Video Super-Resolution Accelerator Exploiting Codec Information Reuse

High-resolution (HR) videos have become popular due to the widespread adoption of high-definition displays. Super-resolution (SR) techniques aim to recover HR frames from low-resolution (LR) frames. Recently, deep neural network (DNN)-based SR methods have achieved superior quality compared to traditional methods. FPGA-based SR accelerators have been proposed to optimize performance and power efficiency. However, most accelerators tailored for video SR only accept uncompressed video frames and operate per-frame DNN inference, ignoring the temporal-spatial information in compressed video bitstreams. In contrast, we observe that non-key frames can be directly constructed using codec information and HR key-frames, saving a significant amount of DNN computing. In this paper, we propose a novel compressed video SR flow and a specific FPGA accelerator called Co-ViSu that integrates decoder, SR, and encoder engines. Co-ViSu exploits codec information reuse scheme to skip non-key frame decoding, avoid complex DNN computation and speed up encoding. Our experimental results show that Co-ViSu achieves 3.6x to 9.4x performance, 4.2x energy efficiency gain with only 0.17dB quality loss compared to the traditional flow, and 2.1x throughput than state-of-the-art.
FLAIRS: FPGA-Accelerated Inference-Resistant &amp; Secure Federated Learning

Federated Learning (FL) has become very popular since it enables clients to train a joint model collaboratively without sharing their private data. However, FL has been shown to be susceptible to backdoor and inference attacks. While in the former, the adversary injects manipulated updates into the aggregation process; the latter leverages clients' local models to deduce their private data. Contemporary solutions to address the security concerns of FL are either impractical for real-world deployment due to high-performance overheads or are tailored towards addressing specific threats, for instance, privacy-preserving aggregation or backdoor defenses. Given these limitations, our research delves into the advantages of harnessing the FPGA-based computing paradigm to overcome performance bottlenecks of software-only solutions while mitigating backdoor and inference attacks. We utilize FPGA-based enclaves to address inference attacks during the aggregation process of FL. We adopt an advanced backdoor-aware aggregation algorithm on the FPGA to counter backdoor attacks. We implemented and evaluated our method on Xilinx VMK-180, yielding a significant speed-up of around 300 times on the IoT-Traffic dataset and more than 506 times on the CIFAR-10 dataset.
Automated Masking of FPGA-Mapped Designs

Due to the importance of FPGAs for secure systems, dealing with private data, protection against side-channel analysis attacks is a must. Although masking is a widely-deployed countermeasure, its application - particularly in hardware - is costly and error-prone. Therefore, generating masked hardware automatically with publicly-available tools such as AGEMA is attractive. As AGEMA was introduced to generate ASIC designs, its direct application on FPGAs is inefficient. In this work, we present AGEMA_FPGA to automatically generate highly-efficient masked circuits for FPGAs. Compared to the original AGEMA designs, our masked FPGA-based circuits utilize up to 64% fewer LUTs and at most 22% fewer FFs while the power consumption is reduced by at most 59%. We further provide an experimental side-channel security analysis of our designs confirming their provable security nature.
Demonstrating NADA: A Workflow for Distributed CNN Training on FPGA Clusters

We introduce our Network Attached Deep learning Accelerator called NADA, which consists of a novel and flexible HW/SW framework for efficient training of deep neural networks on FPGA clusters. NADA is centered around layer parallelism, instantiating a specific implementation for each layer. These implementations are placed across the desired number of network attached FPGAs in the cluster. The NADA hardware architecture relies on a high-speed UDP/IP pure hardware network stack. We demonstrate the usability of our approach by training a couple of demo networks on an Arria 10 FPGA cluster.
MSDF-SGD: Most-Significant Digit-First Stochastic Gradient Descent for Arbitrary-Precision Training

Stochastic gradient descent has been a widely used machine learning algorithm, and interest in low-precision SGD is growing because it improves throughput and keeps efficient convergence. We propose MSDF-SGD, a novel approach allowing SGD to support arbitrary-precision training on FPGAs by employing most-significant digit-first arithmetic. MSDF-SGD is the first architecture that supports arbitrary-precision data, models, and intermediates at the same time. MSDF-SGD is evaluated via training linear classifiers on representative datasets. MSDF-SGD delivers a 1.6× speedup over state-of-the-art low-precision hardware implementations and converges up to 8.6× faster than cutting-edge implementations on CPUs. Finally, we provide a programming interface that permits building a custom arbitrary-precision training accelerator, making MSDF-SGD support more complicated, multi-layered and nonlinear models.
Bayesian Optimization for Efficient Heterogeneous MPSoC Based DNN Accelerator Runtime Tuning

With the explosive growth of Internet of Things (IoT) devices and applications, deploying Deep Neural Networks (DNNs) on resource-constrained embedded edge devices has become a popular research trend. Because such systems have limited resources, they need to rely on optimising resource utilisation to meet performance requirements. However, for scenarios where the DNN application and workloads are dynamically changing, the offline system optimisation technique cannot achieve optimal runtime performance in practical environments. Hence, in this PhD project, we propose a Bayesian Optimisation (BO)-based runtime tuning scheme for improving energy efficiency of heterogeneous MPSoC-based DNN accelerator in the context of DNN applications. By seeking suitable hardware configurations of the accelerator for dynamic DNN inference workloads ranging from 200 M to 600 M FLOPs (floating-point operations) at runtime, the recommended configuration can averagely save up to 15.33% energy consumption from a random configuration setting.
GNNBuilder: An Automated Framework for Generic Graph Neural Network Accelerator Generation, Simulation, and Optimization

There are plenty of graph neural network (GNN) accelerators being proposed. However, they highly rely on users' hardware expertise and are usually optimized for one specific GNN model, making them challenging for practical use. Therefore, in this work, we propose GNNBuilder, the first automated, generic, end-to-end GNN accelerator generation framework. It features four advantages: (1) GNNBuilder can automatically generate GNN accelerators for a wide range of GNN models arbitrarily defined by users; (2) GNNBuilder takes standard PyTorch programming interface, introducing zero overhead for algorithm developers; (3) GNNBuilder supports end-to-end code generation, simulation, accelerator optimization, and hardware deployment, realizing push-button GNN accelerator design; (4) GNNBuilder is equipped with accurate performance models for its generated accelerators, enabling fast and flexible design space exploration (DSE). In the experiments, we show that our accelerator performance model has errors within 36% for latency prediction and 18% for BRAM count prediction. Additionally, we show that our generated accelerators can outperform CPU by 6.33× and GPU by 6.87×. This framework is open-source, and the code is available at https://github.com/sharc-lab/gnn-builder.
A Novel Hardware Accelerator of NeRF Based on Xilinx UltraScale and UltraScale+ FPGA

Neural Radiance Fields (NeRF) has shown its superiority in various fields, including 3D reconstruction and inverse rendering. However, due to its high computational demands, NeRF typically requires implementation on high-performance GPUs, which is costly and power-intensive, thus limiting its applicability. Compared to GPUs, FPGAs offer a potential solution to implement NeRF at lower cost and power. However, FPGA-based NeRF designs are still rare. To address this issue, a novel hardware NeRF accelerator based on Xilinx UltraScale and UltraScale+ FPGA has been proposed. The proposed design is based on the Multiresolution Hash Encoding NeRF algorithm and comprises Feature Reader, Multilayer Perceptrons (MLP), and Volume Renderer. The Feature Reader and MLP calculate the coordinates and directions of the sampling points on the rays, which are then used to determine the colors and densities of these points. To prevent timing problems and routing congestion caused by high DSPs utilization, we use registers and cascade paths between DSPs to construct MACC matrices in MLP. The Volume Renderer processes these colors and densities to obtain the colors of rays. The proposed accelerator runs at 300MHz on Xilinx Alveo U250, achieving an average 1.60 × performance improvement compared to NVIDIA GTX 1080 Ti. Additionally, the accelerator has reduced energy consumption per image by an average of 5.21x and 4.64x compared to NVIDIA GTX 1080 Ti and NVIDIA RTX 3090, respectively.
Calabash: Accelerating Attention Using a Systolic Array Chain on FPGAs

In recent years, attention mechanism has achieved remarkable performance in natural language processing and computer vision applications, at the expense of high computation cost. FPGAs have been demonstrated to be an effective hardware platform for various AI applications. However, the attention mechanism involves complex data dependency, which makes FPGA acceleration difficult. In this paper, we propose Calabash, an FPGA accelerator for attention-based applications. We design a chain of two systolic arrays, applying the same dataflow. Then, we design two scheduling techniques for different matrices to ensure the intermediate matrix can be cached in the on-chip memory. Finally, we develop analytical models for resource utilization estimation, workload balancing, and latency prediction to guide design space exploration. Experiments show that Calabash achieves 1.76 TOP/s, 1.06 TOP/s on Xilinx VU9P and ZCU102 platforms, yielding an average 50.1X and 3.94X energy-efficiency improvement compared with CPU and GPU, respectively.
fpgaHART: A Toolflow for Throughput-Oriented Acceleration of 3D CNNs for HAR onto FPGAs

Surveillance systems, autonomous vehicles, human monitoring systems, and video retrieval are just few of the many applications in which 3D Convolutional Neural Networks are exploited. However, their extensive use is restricted by their high computational and memory requirements, especially when integrated into systems with limited resources. This study proposes a toolflow that optimises the mapping of 3D CNN models for Human Action Recognition onto FPGA devices, taking into account FPGA resources and off-chip memory characteristics. The proposed system employs Synchronous Dataflow (SDF) graphs to model the designs and introduces transformations to expand and explore the design space, resulting in high-throughput designs. A variety of 3D CNN models were evaluated using the proposed toolflow on multiple FPGA devices, demonstrating its potential to deliver competitive performance compared to earlier hand-tuned and model-specific designs.
Challenges Using FPGA Clusters for Distributed CNN Training

While FPGAs are well-established in the field of CNN inference, there is a lack of research regarding CNN training. To mitigate this, we develop an easy-to-use framework for CNN training on network-attached FPGA clusters. Tightly pipelined layer parallelism promises to facilitate a high utilization across a whole cluster. This comes with challenges, which we discuss in this paper.
HPTA: A High Performance Transformer Accelerator Based on FPGA

The transformer neural networks have achieved remarkable performance in both Natural Language Processing (NLP) and Computer Vision (CV) applications, with encoder-decoder architecture based on attention layers. However, implementing transformers on resource-constrained devices presents challenges due to the super-large network structures and nontrivial dataflows. Field-Programmable Gate Arrays (FPGA) have been a promising platform for Neural Network (NN) acceleration due to their design flexibility and customization. Existing FPGA-based implementations of transformers face efficiency and generality issues. This paper proposes HPTA, a high-performance accelerator for implementing transformers on FPGA. We analyze the structural features of transformer networks and design the accelerator with configurable processing element, optimized data selection and arrangement and efficient memory subsystem, to support various transformers. We evaluate the performance of HPTA with BERT and Swin Transformer, the typical transformer models in NLP and CV. HPTA achieves up to 44× and 29× inference time reductions compared with the CPU implementation, and up to 17× and 10x energy efficiency improvements compared with the GPU implementation, for BERT and Swin Transformer, respectively. Compared to the existing FPGA-based accelerators, HPTA shows performance improvements up to 1.3× and 1.8× in inference time compared to NPE and Vis-TOP, respectively.
A refactoring categorization model for software quality improvement

Refactoring, a widely adopted technique, has proven effective in facilitating and reducing maintenance activities and costs. Nonetheless, the effects of applying refactoring techniques on software quality exhibit inconsistencies and contradictions, leading to conflicting evidence on their overall benefit. Consequently, software developers face challenges in leveraging these techniques to improve software quality. Moreover, the absence of a categorization model hampers developers' ability to decide the most suitable refactoring techniques for improving software quality, considering specific design goals. Thus, this study aims to propose a novel refactoring categorization model that categorizes techniques based on their measurable impacts on internal quality attributes. Initially, the most common refactoring techniques used by software practitioners were identified. Subsequently, an experimental study was conducted using five case studies to measure the impacts of refactoring techniques on internal quality attributes. A subsequent multi-case analysis was conducted to analyze these effects across the case studies. The proposed model was developed based on the experimental study results and the subsequent multi-case analysis. The model categorizes refactoring techniques into green, yellow, and red categories. The proposed model, by acting as a guideline, assists developers in understanding the effects of each refactoring technique on quality attributes, allowing them to select appropriate techniques to improve specific quality attributes. Compared to existing studies, the proposed model emerges superior by offering a more granular categorization (green, yellow, and red categories), and its range is wide (including ten refactoring techniques and eleven internal quality attributes). Such granularity not only equips developers with an in-depth understanding of each technique's impact but also fosters informed decision-making. In addition, the proposed model outperforms current studies and offers a more nuanced understanding, explicitly highlighting areas of strength and concern for each refactoring technique. This enhancement aids developers in better grasping the implications of each refactoring technique on quality attributes. As a result, the model simplifies the decision-making process for developers, saving time and effort that would otherwise be spent weighing the benefits and drawbacks of various refactoring techniques. Furthermore, it has the potential to help reduce maintenance activities and associated costs.
Monitoring System for an Experimental Facility Using GMDH Methodology

This work presents a Monitoring System based on the GMDH - Group Method of Data Handling methodology applied in an Experimental Test Facility. GMDH is a combinatorial multi-layer algorithm in which a network of layers and nodes is generated using a number of inputs from the data stream being evaluated. The GMDH method is based on an underlying assumption that the data can be modeled by using an approximation of the Volterra Series or Kolmorgorov-Gabor polynomial. The Fault Test Experimental Facility was designed inspired in a PWR nuclear power plant and is composed by elements that correspond to the pressure vessel, steam generator, pumps of the primary and secondary reactor loops. The nuclear reactor core is represented by an electrical heater with different values of power. The experimental plant is fully instrumented with sensors and actuators. The Fault Test Experimental Facility can be operated to generate normal and faulty data. These failures can be added initially with small magnitude, and their magnitude being increasing gradually in a controlled way. The database will interface with the plant supervisory system SCADA (Supervisory Control and Data Acquisition) that provides the data through standard interface.
Towards ‘up to context’ reasoning about higher-order processes

Proving behavioural equivalences in higher-order languages is a difficult task, because interactions involve complex values, namely terms of the language. In coinductive (i.e., bisimulation-like) techniques for these languages, a useful enhancement is the ‘up-to context’ reasoning, whereby common pieces of context in related terms are factorised out and erased. In higher-order process languages, however, such techniques are rare, as their soundness is usually delicate and difficult to establish. In this paper we adapt the technique of unique solution of equations, that implicitly captures ‘up-to context’ reasoning, to the setting of the Higher-order π-calculus. Equations are written and solved with respect to normal bisimilarity, chosen both because of its efficiency — its clauses do not require universal quantifications on terms supplied by the external observer — and because of the challenges it poses on the ‘up-to context’ reasoning and that already show up when proving its congruence properties.
A Comparative Study Across Static and Dynamic Side-Channel Countermeasures

Timing side-channel attacks remain a major challenge for software security, in particular for cryptographic implementations. Multiple countermeasures against such attacks have been proposed over the last decades, including static and dynamic approaches. Although such countermeasures have been extensively studied in the literature, previous evaluations have mostly relied on simplified system settings. In this article, we provide a comparative evaluation of the effectiveness of both static and dynamic countermeasures in a realistic setting for Java programs. Our experimental setup considers the effects of the non-deterministic timing behavior introduced by the Java VM, in particular involving just-in-time compilation (JIT). Our empirical results indicate that such countermeasures vary heavily on how much they can reduce information leakage, and show that negative effects of non-deterministic timing behavior on their effectiveness are substantial.
A fast and scalable similarity search in high-dimensional image datasets

Owing to the development of image data production and use, the quantity of image datasets has exponentially increased in the last decade. Consequently, the similarity searching cost in image datasets becomes a severe problem which affects the efficiency of similarity search engines in this data type. In this paper, we address the problem of reducing the similarity search cost in large, high-dimensional and scalable image datasets; we propose an improvement of the D-index method to reduce the searching cost and to deal efficiently with scalable datasets. The proposed improvement is based on two propositions; first, we propose criteria and algorithms to choose effective separation values which can reduce the searching cost. Second, we propose an algorithm for updating the structure in case of scalable datasets to resist the impact of objects' insertion on the searching cost. The experiments show that the proposed D-index version has proved a good searching performance in comparison with the classical D-index and a significant resistance to the dataset scalability against the original D-index.
Summary of the International Conference on Software and System Processes (ICSSP 2018)

The International Conference on Software and System Processes (ICSSP), continuing the success of Software Process Workshop (SPW), the Software Process Modeling and Simulation Workshop (ProSim) and the International Conference on Software Process (ICSP) conference series, has become the established premier event in the field of software and systems engineering processes. It provides a leading forum for the exchange of research outcomes and industrial best-practices in process development from software and systems disciplines. ICSSP 2018 was held in Gothenburg, Sweden, 26-27 May 2018, co-located with the 40th International Conference on Software Engineering (ICSE). The theme of ICSSP 2018 was studying \Demands on Processes, Processes on Demand" by recognizing the demands on processes that include the need for both welldeveloped plans and incremental deliveries (agile and hybrid processes), utilization of increased automation (model-based engineering and DevOps), higher degrees of customer collaboration, comprehensive analysis of existing products for reuse (open source and COTS), and performance requirements of enterprise-level architectures. Papers presented at ICSSP discussed these issues across different domains, providing concepts, evidence, and experiences.
No Silver Bullet Reloaded

At XP2017 in Koln, a panel was convened to discuss the classic 1987 IEEE Software paper by Frederick P. Brooks, "No Silver Bullet: Essence and Accidents in Software Engineering." The ideas presented in his paper have influenced several generations of software developers. Brooks emphasized the notions of essential complexity and accidental complexity, and he offered suggestions for promising approaches to software development. While his approaches are linked to what we now recognize as "agile practices," panelists offered an implicit caveat that they must be done with discipline to avoid increased accidental complexity. Panelists also observed that agile development itself is not a "silver bullet".