Reviewerly/exHarmony huggingface dataset:
Knowledge Graph Completing with Dual Confrontation Learning Model based on Variational Information Bottleneck Method

In natural language learning, pre-trained language models (PLM) can acquire rich knowledge and concepts from rich corpora, making it possible to use PLM-based models for knowledge graph completion (KGC) tasks. However, in previous research, when applying pre-trained models to knowledge graph completion tasks, two main challenges persist: (1) Existing knowledge graph completion models are typically evaluated based on the closed-world assumption(CWA), thus lacking evaluation methods suitable for the open-world assumption(OWA), which constitutes a significant challenge in the current field of knowledge graph completion. (2) Extracting useful information, reducing noise, and providing clear interpretability for extracting effective information from the extensive prior knowledge embedded in pre-trained language models is also a crucial issue. Although the loss function can reduce noise to a certain extent, from the perspective of information theory, only relying on the loss function has a limited effect on noise reduction, and the model needs more professional tools to reduce noise and reduce the impact of irrelevant information on model performance. To address the aforementioned challenges, we propose a dual confrontation learning model based on the variational information bottleneck method. This model restricts information flow and feature selection from the perspective of information theory to reduce noise and enhance model performance while providing clear interpretability for this process. Based on extensive experiments and comprehensive evaluations conducted under both closed-world and open-world assumptions, this model successfully extracts valuable knowledge from pre-trained language models to accomplish KGC tasks. Simultaneously, it minimizes noise, removes non-robust features, enhances model reliability, and optimizes model performance. More importantly, we offer a strong interpretability for the process in which our model constrains information flow to reduce noise.
Aster: Encoding Data Augmentation Relations into Seed Test Suites for Robustness Assessment and Fuzzing of Data-Augmented Deep Learning Models

Data-augmented deep learning models are widely used in real-world applications. However, many state-of the-art loss-based or coverage-based fuzzing techniques fail to produce fuzzing samples for them from many seeds. This paper proposes Aster, a novel technique to address this problem to enhance their fuzzing effectiveness for deep learning models trained with multi-sample data augmentation methods. Aster formulates a novel reachability-based strategy to encode the insights of every seed’s direct and indirect data augmentation relation instances into the replacement seed of that seed systematically. Our experiment shows that Aster is highly effective. On average, loss-based and coverage-based fuzzing techniques can generate 166% and 110% more fuzzing samples and reduce 31% and 22% unsuccessful seeds, respectively, after adopting the replacement seeds generated by Aster to replace their original seeds. Their improved models also become up to 55% and 40% on average more robust against FGSM and PGD attacks in the experiment.
Zero Trust Architecture Approach for Developing Mission Critical Embedded Systems

This paper describes a Zero Trust Architecture (ZTA) approach for the survivability development of mission critical embedded systems. Designers could use ZTA as a systems analysis tool to explore the design space. The ZTA concept of “never trust, always verify” is being leveraged in the design process to guide the selection of security and resilience features for the codesign of functionality, performance, and survivability. The design example of a small drone for survivability is described along with the explanation of the ZTA approach.
Mobile Technology as a Catalyst for Enhancing EFL Speaking Skills in Social Language Learning Contexts

Traditional EFL speaking classes have several deficiencies, including a lack of authentic cultural context, limited practice opportunities due to class size and time constraints, and an absence of communicative context. These deficiencies hinder the speaking proficiency of EFL learners. In addition, although mobile-assisted language learning has been recognized as effective, there is a limited focus in existing research on its application to enhance EFL speaking skills within a social language learning environment. This research aimed to address these issues by designing a speaking learning activity for EFL learners using a mobile-assisted social language learning environment. The purpose was to provide enough opportunities and time for learners to practice their speaking skills in communicative, authentic and cultural contexts. The study investigated whether this designed learning activity benefited the learners' EFL speaking performance, including their pronunciation, fluency, vocabulary, grammar, and content. Our research question was as follows: Do language learners who attend the mobile-assisted social language learning activity perform better compared to language learners who attend the traditional language learning activity? Fifty-eight graduate students took part in the study and were randomly assigned to either a control group (who completed learning tasks using the traditional method) or an experimental group (who used a mobile social language learning app). The results indicated that the learners in the experimental group performed significantly better on the EFL speaking post-test than those in the control group. Therefore, it is recommended that the EFL speaking learning activity be conducted within a mobile-assisted social language learning environment as it can enhance learners' speaking abilities.
The Effect of Tilt and Azimuth Angle Variations on Monthly and Annual Incident Solar Radiations for Locations in Brazil

This paper analyzes the effect of the surface's azimuth and tilt angle changes on the incident solar irradiation for every location in Brazil. The monthly and annual incident solar radiations on tilted surfaces under different azimuth and inclination angles are estimated to generate graphs as color contour maps, which show the fraction (%) of the maximum irradiation availability at each location. A simulation tool is developed in Python to generate the graphs by computing the incident solar radiations on inclined surfaces based on the Brazilian global horizontal solar irradiation data and the locations' geographic coordinates. Simulation results show that the changes in azimuth angle did not generally result in significant incident solar radiation losses for the slope angle of up to around 20 degrees. Hence, mounting the PV modules under this tilt angle limitation and any azimuth angle results in maintaining a high level of annual incident radiation. This simulation tool can be used by designers to quickly evaluate the effect of specific tilt and azimuth angles on the monthly and yearly incident solar irradiation for a particular location.
Creating a Dataset for High-Performance Computing Code Translation using LLMs: A Bridge Between OpenMP Fortran and C++

In this study, we present a novel dataset for training machine learning models translating between OpenMP Fortran and C++ code. To ensure reliability and applicability, the dataset is created from a range of representative open-source OpenMP benchmarks. It is also refined using a meticulous code similarity test. The effectiveness of our dataset is assessed using both quantitative (CodeBLEU) and qualitative (human evaluation) methods. We showcase how this dataset significantly elevates the translation competencies of large language models (LLMs). Specifically, models without prior coding knowledge experienced a boost of x 5.1 in their CodeBLEU scores, while models with some coding familiarity saw an impressive x 9.9-fold increase. The best fine-tuned model using our dataset outperforms GPT-4. It is also reaching human-level accuracy. This work underscores the immense potential of our dataset in propelling advancements in the domain of code translation for high-performance computing. The dataset is accessible at https://github.com/bin123apple/Fortran-CPP-HPC-code-translation-dataset.
Eye-Hand Coordination Develops from Active Multimodal Compression

During their first months of life, infants learn to coordinate their perceptions and actions across different modalities. For example, eye-hand coordination relies on combining visual and proprioceptive sensory inputs for controlling eye and hand movements. What drives the development and calibration of such coordination? Here, we put forward a multimodal hierarchical extension of the Active Efficient Coding framework to learn a simple form of eye-hand coordination. By learning to actively compress visual and proprioceptive inputs into a combined multimodal representation, our embodied infant model learns to make eye movements to track an object held in its hand. We find that the abstract multimodal representation improves the tracking accuracy, but only if it emerges after the establishment of the single-modality systems. This suggests the existence of a “less-is-more” effect for the development of coordinated multimodal sensorimotor behaviors.
Opportunistic Query Execution on SmartNICs for Analyzing In-Transit Data

High-performance computing (HPC) systems researchers have proposed using current, programmable network interface cards (or SmartNICs) to offload data management services that would otherwise consume host processor cycles in a platform. While this work has successfully mapped data pipelines to a collection of SmartNICs, users require a flexible means of inspecting in-transit data to assess the live state of the system. In this paper, we explore SmartNIC-driven opportunistic query execution, i.e., enabling the SmartNIC to make a decision about whether to execute a query operation locally (i.e., “offload”) or defer execution to the client (i.e., “push-back”). Characterizations of different parts of the end-to-end query path allow the decision engine to make complexity predictions that would not be feasible by the client alone.
A Multi-Channel Composite Recommendation Method in Federated Learning

There are various data security risks in power grid systems, and recommendation systems are a good solution to the selection of security strategies. Federated learning is a machine learning framework that can protect the data privacy of each participant and can effectively solve the data security external collaborative training modeling challenges. This paper proposed a flexible implantable recommendation algorithm based on knowledge graph and multi-channel composite to enhance federated learning on demand. The algorithm designs three different feature extraction modules to learn to construct multi-dimensional features on multi-attribute collaborative knowledge graphs. Among them, the distance translation module utilizes the Trans R recommendation algorithm to learn the internal semantic structure information in knowledge graph triples and terminal-security policy selection history; the random walk module uses the Node2Vec algorithm to learn the homogeneous and Structured information in the multi-attribute collaborative knowledge graph; the graph convolutional network module learns the high-order neighborhood multi-attribute collaborative information of terminal/security policy nodes through multi-layer feature propagation and aggregation. The composite network module adopts a convolutional neural network-like operation to first locally optimize the three modules, and then extracts the highly informative elements to obtain the final feature representation of the terminal/security policy. Finally, the algorithm merges the vectors of terminal-security policy pairs into the multi-layer perceptron to learn and calculate the prediction score for recommendation.
Deepfakes Signatures Detection in the Handcrafted Features Space

In the Handwritten Signature Verification (HSV) literature, several synthetic databases have been developed for data-augmentation purposes, where new specimens and new identities were generated using bio-inspired algorithms, neuromotor synthesizers, Generative Adversarial Networks (GANs) as well as several deep learning methods. These synthetic databases contain synthetic genuine and forgeries specimens which are used to train and build signature verification systems. Researches on generative data assume that synthetic data are as close as possible to real data, this is why, they are either used for training systems when used for data augmentation tasks or are used to fake systems as synthetic attacks. It is worth, however, to point out the existence of a relationship between the handwritten signature authenticity and human behavior and brain. Indeed, a genuine signature is characterised by specific features that are related to the owner’s personality. The fact which makes signature verification and authentication achievable. Handcrafted features had demonstrated a high capacity to capture personal traits for authenticating real static signatures. We, therefore, Propose in this paper, a handcrafted feature based Writer-Independent (WI) signature verification system to detect synthetic writers and signatures through handcrafted features. We also aim to assess how realistic are synthetic signatures as well as their impact on HSV system’s performances. Obtained results using 4000 synthetic writers of GPDS synthetic database show that the proposed handcrafted features have considerable ability to detect synthetic signatures vs. two widely used real individuals signatures databases, namely CEDAR and GPDS-300, which reach 98.67% and 94.05% of successful synthetic detection rates respectively.
Implementing Jamming Detection on FPGA: An Accelerated Forward Consecutive Mean Excision Approach

Jamming detection is fundamental for improving the stability and spectrum efficiency of wireless communication systems. Although the forward consecutive mean excision (FCME) detection is well-explored from the algorithm perspective, few literature have successfully implemented FCME on the Field Programmable Gate Array (FPGA). In fact, implementing FCME on FPGA is usually laborious due to its complex traversal iteration logic. In this paper, we propose a hardware-friendly detection method called accelerated FCME (AFCME). Our key insight is that the final threshold of FCME can be estimated beforehand, and hence the serial iteration logic of FCME can be bypassed. As a consequence, AFCME can be implemented via the pipeline structure, while provably sharing the identical detection precision with FCME. Experimental results tested on Xilinx XC7K325T FPGA chip hardware platform show performance advantages of the proposed AFCME in terms of stable time- delay, concise control logic and resource usage compared with baseline methods.
Security-as-a-Service for Embedded Systems

In order to promote competition and technology refresh, the Department of Defense (DoD) has required the use of a Modular Open Systems Approach (MOSA) for the acquisition and implementation of embedded systems. Designing security and mission assurance for MOSA systems often relies upon payload specific, potentially inadequate, and difficult to update schemes. Our goal is to develop a physically separated Embedded Security-as-a-Service (ESaaS) function as a payload agnostic, updatable security manager for MOSA systems. In this paper, we describe the ongoing development of an ESaaS module for securing embedded systems that align with DoD standards, OpenVPX and Sensor Open Systems Architecture (SOSA).
Semantic Segmentation using Foundation Models for Cultural Heritage: an Experimental Study on Notre-Dame de Paris

The zero-shot performance of foundation models has captured a lot of attention. Specifically, the Segment Anything Model (SAM) has gained popularity in computer vision due to its label-free segmentation capabilities. Our study proposes using SAM on cultural heritage data, specifically images of Notre-Dame de Paris, with a controlled vocabulary. SAM can successfully identify objects within the cathedral. To further improve segmentation, we utilized Grounding DINO to detect objects and CLIP to automatically add labels from the segmentation masks generated by SAM. Our study demonstrates the usefulness of foundation models for zero-shot semantic segmentation of cultural heritage data.
Identifying Out-of-Domain Objects with Dirichlet Deep Neural Networks

Deep neural networks are usually trained on a closed set of classes, which makes them distrustful when handling previously-unseen out-of-domain (OOD) objects. In safety-critical applications such as perception for automated driving, detecting and localizing OOD objects is crucial, especially if they are positioned in the driving path. In the context of this contribution, OOD objects refer to objects that were not represented in the training dataset. We propose a Dirichlet deep neural network for instance segmentation with inherent uncertainty modeling based on Dirichlet distributions and the Intermediate Layer Variational Inference (ILVI). A thorough analysis shows that our method delivers reliable uncertainty estimates to its predictions whilst identifying OOD instances. The model-agnostic approach can be applied to different instance segmentation models as demonstrated for two different state-of-the-art deep neural networks. Superior results can be shown on the out-of-domain Lost and Found dataset compared to state-of-the-art approaches, whilst also achieving improvements on the in-domain Cityscapes dataset.
Video BagNet: short temporal receptive fields increase robustness in long-term action recognition

Previous work on long-term video action recognition relies on deep 3D-convolutional models that have a large temporal receptive field (RF). We argue that these models are not always the best choice for temporal modeling in videos. A large temporal receptive field allows the model to encode the exact sub-action order of a video, which causes a performance decrease when testing videos have a different sub-action order. In this work, we investigate whether we can improve the model robustness to the sub-action order by shrinking the temporal receptive field of action recognition models. For this, we design Video BagNet, a variant of the 3D ResNet-50 model with the temporal receptive field size limited to 1, 9, 17 or 33 frames. We analyze Video Bag-Net on synthetic and real-world video datasets and experimentally compare models with varying temporal receptive fields. We find that short receptive fields are robust to sub-action order changes, while larger temporal receptive fields are sensitive to the sub-action order.
Channel-Adaptive Dynamic Neural Networks for Low-Complexity Distributed Signal Awareness

Deep neural networks (DNNs) are being increasingly used to achieve signal awareness. Usually, DNNs have been entirely executed at the mobile device. However, as DNNs become more complex, edge computing techniques will become necessary for DNN deployment. Notably, it has been shown that the original DNN architectures can be modified by introducing a "bottleneck" producing a compressed representation of the input. The DNN split computation is then orchestrated between the mobile device and the edge, which respectively execute the layers before and after the bottleneck. As yet, the concept of split computing has not been applied for signal awareness. Moreover, in existing work, the dynamic nature of the wireless channel is not considered in the bottleneck design. In this paper, we propose Split Computing Enhancing Signal Awareness (SpliCES), a new approach where the bottleneck is adapted at inference time according to ongoing channel conditions. Moreover, we present a novel training strategy which optimizes the bottleneck-injected ensemble parameters to generalize to all channel conditions in its first stage, and then specializes each bottleneck separately to a certain level of channel impairment in the second stage. We evaluate SpliCES on the well-known RadioML dataset for modulation recognition, and show that on average, our proposed method transmits 12x less data while experiencing only about 3% accuracy drop in high signal-to-noise ratio (SNR) regimes. With low SNR, SpliCES achieves 5% better accuracy thanks to its dynamic structure.
Decision Boundary Optimization for Few-shot Class-Incremental Learning

Few-shot class-incremental learning (FSCIL) is gaining prominence in real-world machine learning applications, including image classification and face recognition. Existing methods often employ parameter freezing for the backbone and classify based on metric learning. However, these methods suffer from two significant problems. Firstly, training the backbone solely on base classes limits its performance on novel classes due to information loss. Secondly, conventional metric-based strategies for prototype generation tend to introduce confusion in decision boundaries during few-shot tasks. To address these challenges, we propose a novel approach called Decision Boundary Optimization Network (DBONet) for few-shot class-incremental learning. To tackle the first issue, DBONet incorporates an augmentation feature extractor along with a corresponding loss function. This augmentation feature extractor combines samples from different categories to capture richer features. For the second issue, we leverage limited sample representativeness information by introducing the Prototype Generation Module (PGM) into DBONet, enabling the generation of more representative prototypes. The prototypes produced by PGM significantly contribute to the accurate delineation of decision boundaries. Furthermore, we exploit intra-class information to enhance classification precision. Extensive experiments on CIFAR100, miniImageNet, and CUB200 datasets demonstrate that our proposed approach achieves new state-of-the-art results.
Zero-Shot and Few-Shot Video Question Answering with Multi-Modal Prompts

Recent vision-language models are driven by large-scale pretrained models. However, adapting pretrained models on limited data presents challenges such as overfitting, catastrophic forgetting, and the cross-modal gap between vision and language. We introduce a parameter-efficient method to address these challenges, combining multimodal prompt learning and a transformer-based mapping network, while keeping the pretrained models frozen. Our experiments on several video question answering benchmarks demonstrate the superiority of our approach in terms of performance and parameter efficiency on both zero-shot and few-shot settings. Our code is available at https://engindeniz.github.io/vitis.
Shannon Strikes Again! Entropy-based Pruning in Deep Neural Networks for Transfer Learning under Extreme Memory and Computation Budgets

Deep neural networks have become the de-facto standard across various computer science domains. Nonetheless, effectively training these deep networks remains challenging and resource-intensive. This paper investigates the efficacy of pruned deep learning models in transfer learning scenarios under extremely low memory budgets, tailored for TinyML models. Our study reveals that the source task’s model with the highest activation entropy outperforms others in the target task. Motivated by this, we propose an entropy-based Efficient Neural Transfer with Reduced Overhead via PrunIng (ENTROPI) algorithm. Through comprehensive experiments on diverse models (ResNet18 and MobileNet-v3) and target datasets (CIFAR-100, VLCS, and PACS), we substantiate the superior generalization achieved by transfer learning from the entropy-pruned model. Quantitative measures for entropy provide valuable insights into the reasons behind the observed performance improvements. The results underscore ENTROPI’s potential as an efficient solution for enhancing generalization in data-limited transfer learning tasks.