Reviewerly/exHarmony huggingface dataset:
WaveFill: A Wavelet-based Generation Network for Image Inpainting

Image inpainting aims to complete the missing or corrupted regions of images with realistic contents. The prevalent approaches adopt a hybrid objective of reconstruction and perceptual quality by using generative adversarial networks. However, the reconstruction loss and adversarial loss focus on synthesizing contents of different frequencies and simply applying them together often leads to inter-frequency conflicts and compromised inpainting. This paper presents WaveFill, a wavelet-based inpainting network that decomposes images into multiple frequency bands and fills the missing regions in each frequency band separately and explicitly. WaveFill decomposes images by using discrete wavelet transform (DWT) that preserves spatial information naturally. It applies L1 reconstruction loss to the decomposed low-frequency bands and adversarial loss to high-frequency bands, hence effectively mitigate inter-frequency conflicts while completing images in spatial domain. To address the inpainting inconsistency in different frequency bands and fuse features with distinct statistics, we design a novel normalization scheme that aligns and fuses the multi-frequency features effectively. Extensive experiments over multiple datasets show that WaveFill achieves superior image inpainting qualitatively and quantitatively.
ECS-Net: Improving Weakly Supervised Semantic Segmentation by Using Connections Between Class Activation Maps

Image-level weakly supervised semantic segmentation is a challenging task. As classification networks tend to capture notable object features and are insensitive to over-activation, class activation map (CAM) is too sparse and rough to guide segmentation network training. Inspired by the fact that erasing distinguishing features force networks to collect new ones from non-discriminative object regions, we using relationships between CAMs to propose a novel weakly supervised method. In this work, we apply these features, learned from erased images, as segmentation super-vision, driving network to study robust representation. In specifically, object regions obtained by CAM techniques are erased on images firstly. To provide other regions with seg-mentation supervision, Erased CAM Supervision Net (ECS-Net) generates pixel-level labels by predicting segmentation results of those processed images. We also design the rule of suppressing noise to select reliable labels. Our experiments on PASCAL VOC 2012 dataset show that without data annotations except for ground truth image-level labels, our ECS-Net achieves 67.6% mIoU on test set and 66.6% mIoU on val set, outperforming previous state-of-the-art methods.
SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation

Many existing approaches for unsupervised domain adaptation (UDA) focus on adapting under only data distribution shift and offer limited success under additional cross-domain label distribution shift. Recent work based on self-training using target pseudolabels has shown promise, but on challenging shifts pseudolabels may be highly unreliable and using them for self-training may lead to error accumulation and domain misalignment. We propose Selective Entropy Optimization via Committee Consistency (SENTRY), a UDA algorithm that judges the reliability of a target instance based on its predictive consistency under a committee of random image transformations. Our algorithm then selectively minimizes predictive entropy to increase confidence on highly consistent target instances, while maximizing predictive entropy to reduce confidence on highly inconsistent ones. In combination with pseudolabel-based approximate target class balancing, our approach leads to significant improvements over the state-of-the-art on 27/31 domain shifts from standard UDA benchmarks as well as benchmarks designed to stress-test adaptation under label distribution shift. Our code is available at https://github.com/virajprabhu/SENTRY.
Data-driven hybrid petri-net based energy consumption behaviour modelling for digital twin of energy-efficient manufacturing system

Advances in energy-saving technology is main way to achieve carbon neutrality. With the development of digital twin, building the physical-virtual data space for improving energy management capacity of enterprises has received tremendous attention. The energy behaviour model implementing accurate simulation and prediction of energy state is the core meta-model of energy-efficient manufacturing digital twin (EMDT). The widely used state-based energy modelling assumes constant power in operation state and approximately fits the energy behaviour without considering uncertain operation environment, resulting in energy behaviour distortion. A data-driven hybrid petri-net (DDHPN) inspired by both the state-based energy modelling and machine learning was developed for establishing the energy behaviour meta-model. Gaussian kernel extreme learning machine is proposed to fit the instantaneous firing speed of energy consumption continuous transitions in DDHPN. DDHPN-based energy behaviour model is driven by physical data under real-time working conditions, operating parameters, and production load for generating a virtual data space of energy management. Finally, DDHPN was integrated into the EMDT model using unified modelling language. The application in extrusion process and die casting process show that the presented model has higher accuracy in energy behaviour prediction. Furthermore, a digital-twin-based energy management prototype system for extrusion workshop demonstrates its potential.
Context Decoupling Augmentation for Weakly Supervised Semantic Segmentation

Data augmentation is vital for deep learning neural networks. By providing massive training samples, it helps to improve the generalization ability of the model. Weakly supervised semantic segmentation (WSSS) is a challenging problem that has been deeply studied in recent years, conventional data augmentation approaches for WSSS usually employ geometrical transformations, random cropping and color jittering. However, merely increasing the same contextual semantic data does not bring much gain to the networks to distinguish the objects, e.g., the correct image-level classification of "aeroplane" may be not only due to the recognition of the object itself, but also its co-occurrence context like "sky", which will cause the model to focus less on the object features. To this end, we present a Context Decoupling Augmentation (CDA) method, to change the inherent context in which the objects appear and thus drive the network to remove the dependence between object instances and contextual information. To validate the effectiveness of the proposed method, extensive experiments on PASCAL VOC 2012 and COCO datasets with several alternative network architectures demonstrate that CDA can boost various popular WSSS methods to the new state-of-the-art by a large margin. Code is available at https://github.com/suyukun666/CDA
Self-Supervised Learning on Graphs: Contrastive, Generative, or Predictive

Deep learning on graphs has recently achieved remarkable success on a variety of tasks, while such success relies heavily on the massive and carefully labeled data. However, precise annotations are generally very expensive and time-consuming. To address this problem, self-supervised learning (SSL) is emerging as a new paradigm for extracting informative knowledge through well-designed pretext tasks without relying on manual labels. In this survey, we extend the concept of SSL, which first emerged in the fields of computer vision and natural language processing, to present a timely and comprehensive review of existing SSL techniques for graph data. Specifically, we divide existing graph SSL methods into three categories: contrastive, generative, and predictive. More importantly, unlike other surveys that only provide a high-level description of published research, we present an additional mathematical summary of existing works in a unified framework. Furthermore, to facilitate methodological development and empirical comparisons, we also summarize the commonly used datasets, evaluation metrics, downstream tasks, open-source implementations, and experimental study of various algorithms. Finally, we discuss the technical challenges and potential future directions for improving graph self-supervised learning. Latest advances in graph SSL are summarized in a GitHub repository <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/LirongWu/awesome-graph-self-supervised-learning</uri> .
LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement

Underwater image enhancement is an important low-level computer vision task for autonomous underwater vehicles and remotely operated vehicles to explore and understand the underwater environments. Recently, deep convolutional neural networks (CNNs) have been successfully used in many computer vision problems, and so does underwater image enhancement. There are many deep-learning-based methods with impressive performance for underwater image enhancement, but their memory and model parameter costs are hindrances in practical application. To address this issue, we propose a lightweight adaptive feature fusion network (LAFFNet). The model is the encoder-decoder model with multiple adaptive feature fusion (AAF) modules. AAF subsumes multiple branches with different kernel sizes to generate multi-scale feature maps. Furthermore, channel attention is used to merge these feature maps adaptively. Our method reduces the number of parameters from 2.5M to 0.15M (around 94% reduction) but outperforms state-of-the-art algorithms by extensive experiments. Furthermore, we demonstrate our LAFFNet effectively improves high-level vision tasks like salience object detection and single image depth estimation.
A blockchain-based approach to smart cargo transportation using UHF RFID

Today, the continuous growth of electronic commerce volume increases the importance of point-to-point shipment transportation. For this reason, it is extremely important that shipment management systems provide effective, efficient and fast service. A study of the literature shows that the improvements made in the field of transportation have been generally aimed at global logistics networks. In this study, unlike the existing literature, a blockchain-based solution is developed for local cargo networks. With this approach using Ultra High Radio Frequency (UHF-RFID), Internet of Things (IoT) sensors and blockchain-based smart contracts, a fast shipping management architecture that ensures security between the parties is provided. In this context, an automatic payment and approval mechanism is established between the parties using ethereum-based smart contracts. In addition, shipments equipped with RFID tags are automatically tracked and traceable using UHF-RFID antennas. The IoT data collected in the study are stored on a cloud-based server, which is a cost-effective solution. Smart contracts that enable communication between the parties are written in the Solidity language using the Ganache, Truffle and Metamask platforms. It was observed that the integration between the technologies is ensured and the communication between the parties works successfully in the test processes.
StrucTexT: Structured Text Understanding with Multi-Modal Transformers

Structured text understanding on Visually Rich Documents (VRDs) is a crucial part of Document Intelligence. Due to the complexity of content and layout in VRDs, structured text understanding has been a challenging task. Most existing studies decoupled this problem into two sub-tasks: entity labeling and entity linking, which require an entire understanding of the context of documents at both token and segment levels. However, little work has been concerned with the solutions that efficiently extract the structured data from different levels. This paper proposes a unified framework named StrucTexT, which is flexible and effective for handling both sub-tasks. Specifically, based on the transformer, we introduce a segment-token aligned encoder to deal with the entity labeling and entity linking tasks at different levels of granularity. Moreover, we design a novel pre-training strategy with three self-supervised tasks to learn a richer representation. StrucTexT uses the existing Masked Visual Language Modeling task and the new Sentence Length Prediction and Paired Boxes Direction tasks to incorporate the multi-modal information across text, image, and layout. We evaluate our method for structured text understanding at segment-level and token-level and show it outperforms the state-of-the-art counterparts with significantly superior performance on the FUNSD, SROIE, and EPHOIE datasets.
Industrial ontologies for interoperability in agile and resilient manufacturing

Ontologies provide an opportunity to tackle the interoperability challenge in digital manufacturing. Although ontologies have been used in numerous industrial projects, the problem of semantic interoperability in the industry is still unresolved. One reason for the lack of success of ontologies in the industry is that most industrial ontologies have been developed based on local needs without taking wider applications into account or attempting to learn from earlier ontology initiatives or reuse the existing ontologies. In this position paper, we argue that for widespread adoption and implementation of ontologies, more systematic and coordinated efforts are needed with the participation of industry, academia, and government at an international level. We also hypothesise that a coordinated and systematic ontology development effort will incrementally lead to the creation of an ecosystem of interoperable software applications that support consistent data access and reasoning across the product life cycle. We discuss why efforts such as Industrial Ontology Foundry (IOF) can promote a methodological ontology development process by introducing modular, and reusable ontologies. We draw examples from two domain areas, namely supply chain management and production planning and scheduling, to demonstrate how industrial notions can be analysed ontologically according to IOF procedures and methods.
Federated Intrusion Detection in Blockchain-Based Smart Transportation Systems

With the integration of the Internet of Things (IoT) in the field of transportation, the Internet of Vehicles (IoV) turned to be a vital method for designing Smart Transportation Systems (STS). STS consist of various interconnected vehicles and transportation infrastructure exposed to cyber intrusion due to the broad usage of software and the initiation of wireless interfaces. This study proposes a federated deep learning-based intrusion detection framework (FED-IDS) to efficiently detect attacks by offloading the learning process from servers to distributed vehicular edge nodes. FED-IDS introduces a context-aware transformer network to learn spatial-temporal representations of vehicular traffic flows necessary for classifying different categories of attacks. Blockchain-managed federated training is presented to enable multiple edge nodes to offer secure, distributed, and reliable training without the need for centralized authority. In the blockchain, miners confirm the distributed local updates from participating vehicles to stop unreliable updates from being deposited on the blockchain. The experiments on two public datasets (i.e., Car-Hacking, TON&#x005F;IoT) demonstrated the efficiency of FED-IDS against state-of-the-art approaches. It reveals the credibility of securing networks of intelligent transportation systems against cyber-attacks.
Hermes

Federated learning (FL) has been a popular method to achieve distributed machine learning among numerous devices without sharing their data to a cloud server. FL aims to learn a shared global model with the participation of massive devices under the orchestration of a central server. However, mobile devices usually have limited communication bandwidth to transfer local updates to the central server. In addition, the data residing across devices is intrinsically statistically heterogeneous (i.e., non-IID data distribution). Learning a single global model may not work well for all devices participating in the FL under data heterogeneity. Such communication cost and data heterogeneity are two critical bottlenecks that hinder from applying FL in practice. Moreover, mobile devices usually have limited computational resources. Improving the inference efficiency of the learned model is critical to deploy deep learning applications on mobile devices. In this paper, we present Hermes - a communication and inference-efficient FL framework under data heterogeneity. To this end, each device finds a small subnetwork by applying the structured pruning; only the updates of these subnetworks will be communicated between the server and the devices. Instead of taking the average over all parameters of all devices as conventional FL frameworks, the server performs the average on only overlapped parameters across each subnetwork. By applying Hermes, each device can learn a personalized and structured sparse deep neural network, which can run efficiently on devices. Experiment results show the remarkable advantages of Hermes over the status quo approaches. Hermes achieves as high as 32.17% increase in inference accuracy, 3.48× reduction on the communication cost, 1.83× speedup in inference efficiency, and 1.8× savings on energy consumption.
Self-adaptive resources allocation-based differential evolution for constrained evolutionary optimization

When using evolutionary algorithms to address constrained optimization problems, it is important to balance not only the diversity and convergence but also the constraints and objective function. To this end, a self-adaptive resources allocation-based differential evolution (SRADE) is presented in this paper. Specifically, during the evolutionary process, three mutation strategies with distinct focuses are collaboratively employed and adaptively assigned to different individuals based on their performance feedback. That is, most of the computing resources will be consumed by the most efficient strategy at different evolutionary stages to mitigate inefficient search under limited resources. These three collaborative strategies focus on maintaining population diversity, driving the population into feasible regions, and promoting the population toward the objective, respectively. Combining the self-adaptive resources allocation scheme and diverse search strategies is expected to satisfy the requirements of the population for diversity, convergence, constraints, and the objective at a certain iteration. Extensive experiments are performed on three benchmark test suites, including a large number of test functions from IEEE CEC 2006, 2010, and 2017. Compared to other well-designed constrained evolutionary approaches, SRADE exhibits superior or very competitive performance.
Educational Blockchain: A Secure Degree Attestation and Verification Traceability Architecture for Higher Education Commission

Degree attestation verification and traceability are complex one-to-one processes between the Higher Education Commission (HEC) and universities. The procedure shifted to the digitalized manner, but still, on a certain note, manual authentication is required. In the initial process, the university verified the degree and stamp seal first. Then, a physical channel of degree submission to the receiving ends is activated. After that, the degree is attested while properly examining and analyzing the tamper records related to degree credentials through e-communication with the university for verification and validation. This issue poses a serious challenge to educational information integrity and privacy. Potentially, blockchain technology could become a standardized platform to perform tasks including issuing, verifying, auditing, and tracing immutable records, which would enable the HEC, universities, and Federal Education Ministry (FEM) to quickly and easily get attested and investigate the forge proof versions of certificates. Besides, decentralized distributed data blocks in chronological order provide high security between distributed ledgers, consensus engine, digital signature, smart contracts, permissioned application, and private network node transactions that guarantee degree record validation and traceability. This paper presents an architecture (HEDU-Ledger) and detail design of blockchain-enabled hyperledger fabric applications implementation for degree attestation verification and traceable direct channel design between HEC and universities. The hyperledger fabric endorses attestation records first, and then validates (committer) the degree and maintains the secure chain of tracing between stakeholder peer nodes. Furthermore, this HEDU-Ledger architecture avoids language and administrative barriers. It also provides robustness in terms of security and privacy of records and maintains integrity with secure preservation as compared to that of the other state-of-the-art methods.
An effective ensemble deep learning framework for text classification

Over the last decade Deep learning-based models surpasses classical machine learning models in a variety of text classification tasks. The primary challenge with text classification is determining the most appropriate deep learning classifier. Numerous research initiatives incorporated ensemble learning to boost the performance, minimize errors and avoid overfitting. However, the performance of the ensemble-methods is limited by the baseline classifiers and the fusion method. The current study makes the following contributions: First, it proposes a new meta-learning ensemble method that fuses baseline deep learning models using 2-tiers of meta-classifiers. Second, it conducts several experiments on six public benchmark datasets to evaluate the performance of the proposed ensemble. For each benchmark dataset, committees of different deep baseline classifiers are trained, and their best performance is compared with the performance of the proposed ensemble. Furthermore, the paper extends the results by comparing the performance of the proposed ensemble method to other state-of-the-art ensemble methods. The findings indicate that the proposed ensemble method significantly improve the classification accuracy of the baseline deep models. Furthermore, the proposed method outperforms the state-of-art ensemble methods. Finally, using the probability distributions for each class label of the deep baseline models improves the performance of the proposed ensemble method.
Machine learning for encrypted malicious traffic detection: Approaches, datasets and comparative study

As people's demand for personal privacy and data security becomes a priority, encrypted traffic has become mainstream in the cyber world. However, traffic encryption is also shielding malicious and illegal traffic introduced by adversaries, from being detected. This is especially so in the post-COVID-19 environment where malicious traffic encryption is growing rapidly. Common security solutions that rely on plain payload content analysis such as deep packet inspection are rendered useless. Thus, machine learning based approaches have become an important direction for encrypted malicious traffic detection. In this paper, we formulate a universal framework of machine learning based encrypted malicious traffic detection techniques and provided a systematic review. Furthermore, current research adopts different datasets to train their models due to the lack of well-recognized datasets and feature sets. As a result, their model performance cannot be compared and analyzed reliably. Therefore, in this paper, we analyse, process and combine datasets from 5 different sources to generate a comprehensive and fair dataset to aid future research in this field. On this basis, we also implement and compare 10 encrypted malicious traffic detection algorithms. We then discuss challenges and propose future directions of research.