Reviewerly/exHarmony huggingface dataset:
EENet: Energy Efficient Neural Networks with Run-time Power Management

Deep learning approaches, such as convolution neural networks (CNNs), have achieved tremendous success in versatile applications. However, one of the challenges to deploy the deep learning models on resource-constrained systems is its huge energy cost. As a dynamic inference approach, early exit adds exiting layers to the networks, which can terminate the inference earlier with accurate results to save energy. The current passive decision-making for energy regulation of early exit cannot adapt to ongoing inference status, varying inference workloads, and timing constraints, let alone guide the reasonable configuration of the computing platforms alongside the inference proceeds for potential energy saving. In this paper, we propose an Energy Efficient Neural Networks (EENet), which introduces a plug-in module to the state-of-the-art networks by incorporating run-time power management. Within each inference, we establish prediction of where the network will exit and adjust computing configurations (i.e., frequency and voltage) accordingly over a small timescale. Considering multiple inferences over a large timescale, we provide frequency and voltage calibration advice, given inference workloads and timing constraints. Finally, the dynamic voltage and frequency scaling (DVFS) governor configures voltage and frequency to execute the network according to the prediction and calibration. Extensive experimental results demonstrate that EENet achieves up to 63.8% energy-saving compared with classic deep learning networks and 21.5% energy-saving compared with the early exit under state-of-the-art exiting strategies, together with improved timing performance.
Range-Invariant Approximation of Non-Linear Operations for Efficient BERT Fine-Tuning

This paper proposes a range-invariant approximation of non-linear operations for training computations of Transformer-based large language models. The proposed method decomposes the approximation into the scaling and the range-invariant resolution for LUT approximation, covering diverse data ranges of non-linear operations with drastically reduced LUT entries during task-dependent BERT fine-tuning. We demonstrate that the proposed method robustly approximates all the non-linear operations of BERT without score degradation on challenging GLUE benchmarks using only a single-entry LUT, facilitating 52% area savings in hardware implementation.
Improved DeepSORT-based pedestrian tracking and recognition method with fused face information

In order to solve the problem of continuity and accuracy of regional pedestrian identification, a method for regional identification and tracking of specific pedestrians with fusion of face information is proposed by using the front and rear frame information of the video stream. The FaceNet face identity network is constructed using the lightweight MobilenetV3-Large feature extraction network, and a fully connected layer is added for classification. Cross-Entropy Loss and Triplet Loss are integrated to build a classifier for stable loss convergence. The identity recognition method is introduced into DeepSORT, and an identity constraint module is added to solve the IDswitch caused by changes in appearance or pedestrian direction. This method combines the two modules of identification network and pedestrian tracking, and then realizes the continuous identification and tracking of pedestrians in the area. The experimental results show that: in terms of identity recognition, the accuracy of the improved system is 11% higher than that of the original algorithm in the autonomous test link; the accuracy of the tracking model is 69.S3% in MOTA on the MOT16 dataset, and the improved system is compared with the original algorithm in the autonomous test. During the test, MOTA increased from 72.38% to 89.84%, while IDswitch was greatly reduced. It realizes the detection and continuous tracking of specific pedestrians in the area, and provides technical support for the behavior of pedestrians in the area.
TD-Magic: From Pictures of Timing Diagrams To Formal Specifications

We introduce TD-Magic, the first neuro-symbolic approach for translating an image of a timing-diagram (TD) to a formal specification. We overcome the lack of labelled data for supervised learning, by first developing a synthetic data generator of labelled TDs. We then use object detection techniques to identify rising and failing edges, OCR to recognise the text, and image processing algorithms to capture synchronisation patterns. Finally, we use semantic interpretation to analyse the extracted features and generate the associated formal specification. Our experiments on industrial TDs show high translation accuracy opening the way to more sophisticated requirements-extraction algorithms from pictures.
AdaGL: Adaptive Learning for Agile Distributed Training of Gigantic GNNs

Distributed GNN training on contemporary massive and densely connected graphs requires information aggregation from all neighboring nodes, which leads to an explosion of inter-server communications. This paper proposes AdaGL, a highly scalable end-to-end framework for rapid distributed GNN training. AdaGL novelty lies upon our adaptive-learning based graph-allocation engine as well as utilizing multi-resolution coarse representation of dense graphs. As a result, AdaGL achieves an unprecedented level of balanced server computation while minimizing the communication overhead. Extensive proof-of-concept evaluations on billion-scale graphs show AdaGL attains ∼30−40% faster convergence compared with prior arts.
CSQ: Growing Mixed-Precision Quantization Scheme with Bi-level Continuous Sparsification

Mixed-precision quantization has been applied on deep neural networks (DNNs) as it leads to significantly better efficiency-accuracy tradeoffs compared to uniform quantization. Meanwhile, determining the exact precision of each layer remains challenging. Previous attempts on bit-level regularization and pruning-based dynamic precision adjustment during training suffer from noisy gradients and unstable convergence. In this work, we propose Continuous Sparsification Quantization (CSQ), a bit-level training method to search for mixed-precision quantization schemes with improved stability. CSQ stabilizes the bit-level mixed-precision training process with a bi-level gradual continuous sparsification on both the bit values of the quantized weights and the bit selection in determining the quantization precision of each layer. The continuous sparsification scheme enables fully-differentiable training without gradient approximation while achieving an exact quantized model in the end. A budget-aware regularization of total model size enables the dynamic growth and pruning of each layer’s precision towards a mixed-precision quantization scheme of the desired size. Extensive experiments show CSQ achieves better efficiency-accuracy tradeoff than previous methods on multiple models and datasets.
Condense: A Framework for Device and Frequency Adaptive Neural Network Models on the Edge

With the popularity of battery-powered edge computing, an important yet under-explored problem is the supporting of DNNs for diverse edge devices. On the one hand, different edge platforms have various runtime requirements and computation/memory capabilities. Deploying the same DNN model is unsatisfiable, while designing a specialized DNN for each platform is prohibitively expensive. On the other hand, for a single edge device, DVFS is leveraged to prolong the battery, incurring significant inference speed variation for the same DNN and consequently poor user experience. To tackle this, we propose Condense, a framework providing a single adaptive model that can be reconfigured (switch to various sub-networks with different computations/parameters) instantly for diverse devices and execution frequencies without any retraining. Experiments demonstrate that Condense can simultaneously provide vast high-accuracy sub-networks with different computations and parameters corresponding to various sparsity ratios to support diverse edge devices with different runtime requirements, and reduce the speed variation under varying frequencies on each device, with a memory cost of only one set of weights.
Development and Experimental Study of a Self-Aware Posttensioned Prestressed Intelligent Reinforcement Structure

A self-aware posttensioned prestressing intelligent reinforcement system with force and sensors is proposed by combining the advantages of posttensioned prestressing reinforcement technology and intelligent structure. A static test study is then conducted with brick masonry members (wall pieces) as reinforcement objects. The experimental results demonstrate that the intelligent strand can be deployed using the same process, tensioning apparatus, and anchoring equipment as ordinary strand. Furthermore, the mechanical performance indexes with and without intelligent strands are very close, and the differences in percentages of cracking load, cracking displacement, yielding load, yielding displacement, ultimate load, and ultimate displacement are 20%, 9%, 9%, 10%, 2%, and 26%.
Girth of the algebraic bipartite graph D(k,q)

For integer k≥2 and prime power q, the algebraic bipartite graph D(k,q) proposed by Lazebnik and Ustimenko in 1993 is useful not only in extremal graph theory but also in coding theory and cryptography. This graph is q-regular, edge-transitive and of girth at least k+4. Its exact girth g=g(D(k,q)) was conjectured in 1995 to be k+5 for odd k and q≥4. This conjecture was shown to be valid in 2016 when k+52|p(q−1), where p is the characteristic of Fq and m|pn means that m divides prn for some nonnegative integer r. In this paper, for t≥1, we prove that (a) g(D(4t+2,q))=g(D(4t+1,q)); (b) g(D(4t+3,q))=4t+8 if g(D(2t,q))=2t+4; (c) g(D(8t,q))=8t+4 if g(D(4t−2,q))=4t+2; (d) g(D(2s+2t−5,q))=2s+2t if p≥3, 2s|(q−1), 2s+1∤(q−1), 2∤t and t|p(q−1). A simple upper bound for the girth of D(k,q) is proposed in the end of this paper.
Physically Feasible Repair of Reactive, Linear Temporal Logic-Based, High-Level Tasks

A typical approach to creating complex robot behaviors is to compose atomic controllers, or skills, such that the resulting behavior satisfies a high-level task; however, when a task cannot be accomplished with a given set of skills, it is difficult to know how to modify the skills to make the task possible. We present a method for combining symbolic repair with physical feasibility checking and implementation to automatically modify existing skills such that the robot can execute a previously infeasible task. We encode robot skills in linear temporal logic (LTL) formulas that capture both safety constraints and goals for reactive tasks. Furthermore, our encoding captures the full skill execution, as opposed to prior work where only the state of the world before and after the skill is executed are considered. Our repair algorithm suggests symbolic modifications, then attempts to physically implement the suggestions by modifying the original skills subject to Linear Temporal Logic (LTL) constraints derived from the symbolic repair. If skills are not physically possible, we automatically provide additional constraints for the symbolic repair. We demonstrate our approach with a Baxter and a Clearpath Jackal.
An Enhanced Regularized Clustering Method With Adaptive Spurious Connection Detection

The regularized clustering (RC) framework based on the fusion penalty has attracted extensive attention in the last decade because it does not require the prior knowledge of the number of clusters. Although the ground truth connections and weights among samples are beneficial for clustering, the performance of RC could be distorted by the omnipresent spurious connections in real-world data sets. To effectively address the issue, this letter constructs an enhanced regularized clustering model by incorporating a spurious connection detection mechanism into the objective function of the RC framework. The proposed model can effectively reduce the damage caused by spurious connections through adaptively identifying the importance of each connection. Furthermore, an alternating minimization algorithm is developed with detailed convergence analysis. Experimental results validate its effectiveness against several state-of-the-art RC methods
Region Selective Fusion Network for Robust RGB-T Tracking

RGB-T tracking utilizes thermal infrared images as a complement to visible light images in order to perform more robust visual tracking in various scenarios. However, the highly aligned RGB-T image pairs introduces redundant information, the modal quality fluctuation during tracking also brings unreliable information. Existing RGB-T trackers usually use channelwise multi-modal feature fusion in which the low-quality features degrades the fused features and causes trackers to drift. In this work, we propose a region selective fusion network that first evaluates each image region by cross-modal and cross-region modeling, then removes low-quality redundant region features to alleviate the negative effects caused by unreliable information in multi-modal fusion. Besides, the region removal scheme brings a efficiency boost as redundant features are removed progressively, this enables the tracker to run at a high tracking speed.Extensive experiments show that the proposed tracker achieves competitive performance with a real-time tracking speed on multiple RGB-T tracking benchmarks including LasHeR, RGBT234 and GTOT.
A Spatio-Temporal Video Denoising Co-Processor With Adaptive Codec

With the increasing demand for high-resolution video and real-time processing, the limited efficiency of video-denoising algorithms has become a critical factor. This paper proposes a spatio-temporal video denoising co-processor to suppress an image sequence’s spatial and temporal noise. Temporal denoising is achieved by merging the current and previous frames at the pixel level in which the current frame is processed by a spatial filter. After exploiting noise estimation and motion detection, the Wiener filter calculates the merge ratio. Rather than buffering the entire previous frame, the JPEG-like codec can dynamically adjust the compression ratio through a predefined quantization table to satisfy the designed on-chip storage. The experimental results demonstrate that the spatio-temporal denoising co-processor can effectively eliminate the fluctuation of the grayscale value of the noise in videos. Simultaneously, the adaptive codec can reduce the storage space consumption for the frame buffer by at least 80% of the original size. To the best of our knowledge, this is the first fully integrated spatio-temporal denoising co-processor without any external memory. Additionally, the grayscale, RGB, and RAW versions of the co-processor are also implemented on the Stratix V FPGA platform and synthesized in 28nm CMOS technology.
Demonstration of Error-Tolerant Integrated Optical Processors Based on Multi-Plane Light Conversion

An integrated optical unitary processor (OUP) that can perform arbitrary unitary mode conversions is expected to be a key device in diverse applications, including optical communication, deep learning, and quantum computing. While integrated OUPs have been experimentally demonstrated, most of them are sensitive to fabrication errors because of their stringent requirements on the splitting ratio of optical splitters. Here, we experimentally demonstrate that the OUP based on multi-plane light conversion (MPLC) manifests excellent robustness against various deviations in the waveguide geometries and the operation wavelength. By comparing various silicon photonic <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$4\times 4$ </tex-math></inline-formula> MPLC-based OUPs operated in the intensity-only mode, we show that similar performances are obtained in all cases under large waveguide geometry deviations and wavelength shifts, which would significantly degrade the performance of conventional OUPs.
Type 2 Fuzzy Neural Controller for Navigation Control of an Ackermann Steering Vehicle

In this study, we proposed a type 2 fuzzy neural controller (FNC) based on Bayesian dynamic group particle swarm optimization (BDGPSO) for the navigation control of Ackermann steering vehicles. The type 2 FNC has a five-layer network architecture. The advantages of the proposed BDGPSO algorithm are that it prevents PSO from falling into a local optimum, and it uses the Bayesian algorithm to determine the optimal inertia weight and learning factor combination. To evaluate the proposed type 2 FNC on the basis of the distance information returned by lidar, we used a new fitness function from the reinforcement-learning strategy. In an unknown testing environment, the fitness value and time required for the proposed type 2 FNC with angular and linear velocity outputs were 0.973510 and 25.555 s, respectively. These values indicated that the proposed approach outperforms other wall-following control methods. In addition, the proposed controller could successfully control the navigation of an Ackerman steering vehicle in an unknown actual environment.
EOG-Based Reading Detection in the Wild Using Spectrograms and Nested Classification Approach

Electrooculography, also known as EOG, is a technique that is used to calculate the corneo-retinal standing potential, which is located between the cornea and the retina of the human eye. Applications of EOG include eye disease diagnosis and eye movement tracking. There has been various research on reading activity detection from EOG signals in controlled or laboratory settings. However, determining reading behaviours from data collected from real-world environments remains a challenging problem. Reading detection in practical scenarios can lead us to track our daily reading activity, in turn improving our learning experience and even workplace productivity. Tracking regular reading behaviour can also lead to further research in cognitive psychology, literacy development, reading motivation, and reading comprehension. In this study, we investigated an electrooculogram dataset that was collected on the field from 10 users who were engaged in their daily activities on two separate days. We propose a pipeline combining the statistical features with deep learning features from pre-trained ImageNet models. To detect the fine-grained reading activities, we employed a nested classification approach where we detect reading and not reading at first and then do one more step of classification to discriminate among three different reading activities. With our pipeline, we could achieve 66.56% accuracy in detecting the reading activities whereas the original dataset publication showed a baseline performance of only 32%.
Is Formal Verification of seL4 Adequate to Address the Key Security Challenges of Kernel Design?

Formal method tools are used in the initial stages of the software development cycle and have advanced in to deal with the design difficulties related to ensuring strong cybersecurity an reliability in high-assurance systems. Operating system kernels are the security keystone of most computer systems. Their continuous advances require formal verification that guarantees the accuracy of their functionalities. As the world’s first microkernel to be proven mathematically secure and functionally correct, seL4 has been adopted for use in a range of critical systems, including defense, aerospace, and financial services applications. In spite of the great effort of the seL4 team to present a comprehensive formal verification of the kernels, there are some security aspects of verification that suffer from a limited scope. From an outsider’s perspective, this paper aims to evaluate if seL4 formal verification is adequate to address security requirements and surveys the recent state of the art for seL4 microkernels.
Hyper-Progressive Single Shot Detector (HPSSD) Algorithm for Door Panel Type-B Detection

In an automobile, the door panel type-B constitutes the interior compartment of the door, primarily composed of screws and white installations forming its structural framework. However, automated manufacturing and maintenance procedures often struggle to accurately detect these components due to their pronounced resemblance to other elements on the panel. Computer vision techniques present a viable solution to this challenge. In this paper, we propose the Hyper-Progressive Single Shot Detector (HPSSD), an object detection algorithm designed to address the aforementioned challenge. Our proposed HPSSD builds on the Single Shot Detector (SSD) algorithm and introduces several enhancements to improve its detection capabilities. The first modification involves replacing the VGG-16 backbone with a ResNet-50 module. Furthermore, we incorporated the Residual Convolutional Block Attention Mechanism (RCBAM) to boost the algorithm’s functionality. To enlarge the receptive fields of each pixel—an essential step for enhancing detection accuracy—we executed multi-dilated convolutions. In the final stage of our development process, we embedded a three-stage progressive attention mechanism (PAM). The PAM is instrumental in generating refined feature maps, which serve as the foundation for precise object detection on the door panel dataset comprising 1200 images. After running 50k iterations on the door panel dataset, the HPSSD displayed a promising mean average precision of 98.2% at a speed of 21 frames per second (FPS). Our results suggest that the HPSSD, with its ability to deliver real-time, accurate detection, is an ideal tool for improving the quality inspection of door panels in smart factories.
A Hierarchical Approach for Improved Anomaly Detection in Video Surveillance

Anomaly detection for video surveillance gains more attention as the number of deployed cameras constantly increases while the state-of-the-art (SOTA) machine learning methods push the detection performance to its limit. Low complexity methods are relatively straightforward to train (low variance) but suffer from high bias (low performance) whereas, the complex ones can achieve high performance (low bias) with a large sample size to suppress the high variance of estimated parameters. Also, most of the SOTA methods can only detect indigenous anomalies that are spatially stationary, failing at detecting the locational anomalies that are due to nonstationary spatial statistics. To solve these issues, we propose an ensemble technique based on a context tree that generates a hierarchical ensemble of image plane partitions, which we call context tree based anomaly detection (CTBAD). With CTBAD, partitions yield anomaly detection models of varying complexities, i.e., from coarse to fine details in partitioning with each partition model (which can be any SOTA method) trained separately to allow the detection of locational anomalies, and then we combine them linearly in a weighted manner to achieve a gradual transition from simpler models to more complex ones as more data become available in a video stream. As a result, CTBAD benefits from low variance of low complexity methods when the data is sparse and exploits high complexity to achieve low bias when sufficient data is observed. Our experiments show that we significantly reduce the number of training samples to reach the same accuracy of a complex model while successfully detecting the locational anomalies.