Reviewerly/exHarmony huggingface dataset:
Blockchain-Based Water-Energy Transactive Management With Spatial-Temporal Uncertainties

Water resources are vital to the energy conversion process but few efforts have been devoted to the joint optimization problem which is fundamentally critical to the water-energy nexus for small-scale or remote energy systems (e.g., energy hubs). Traditional water and energy trading mechanisms depend on centralized authorities and cannot preserve security and privacy effectively. Also, their transaction process cannot be verified and is subject to easy tampering and frequent exposures to cyberattacks, forgery, and network failures. Toward that end, water-energy hubs (WEHs) offers a promising way to analyse water-energy nexus for greater resource utilization efficiency. We propose a two-stage blockchain-based transactive management method for multiple, interconnected WEHs. Our method considers peer-to-peer (P2P) trading and demand response, and leverages blockchain to create a secure trading environment. It features auditing and resource transaction record management via system aggregators enabled by a consortium blockchain, and entails spatial-temporal distributionally robust optimization (DRO) for renewable generation and load uncertainties. A spatial-temporal ambiguity set is incorporated in DRO to characterize the spatial-temporal dependencies of the uncertainties in distributed renewable generation and load demand. We conduct a simulation-based evaluation that includes robust optimization and the moment-based DRO as benchmarks. The results reveal that our method is consistently more effective than both benchmarks. Key findings include i) our method reduces conservativeness with lower WEH trading and operation costs, and achieves important performance improvements by up to 6.1%; and ii) our method is efficient and requires 18.7% less computational time than the moment-based DRO. Overall, this study contributes to the extant literature by proposing a novel two-stage blockchain-based WEH transaction method, developing a realistic spatial-temporal ambiguity set to effectively hedge against the uncertainties for distributed renewable generation and load demand, and producing empirical evidence suggesting its greater effectiveness and values than several prevalent methods.
Recovering Rainbow’s Secret Key with a First-Order Fault Attack

Rainbow, a multivariate digital signature scheme and third round finalist in NIST's PQC standardization process, is a layered version of the unbalanced oil and vinegar (UOV) scheme. We introduce two fault attacks, each focusing on one of the secret linear transformations T and S used to hide the structure of the central map in Rainbow. The first fault attack reveals a part of T and we prove that this is enough to achieve a full key recovery with negligible computational effort for all parameter sets of Rainbow. The second one unveils S, which can be extended to a full key recovery by the Kipnis-Shamir attack. Our work exposes the secret transformations used in multivariate signature schemes as an important attack vector for physical attacks, which need further protection. Our attacks target the optimized Cortex-M4 implementation and require only first-order instruction skips and a moderate amount of faulted signatures.
KeyTr: Keypoint Transporter for 3D Reconstruction of Deformable Objects in Videos

We consider the problem of reconstructing the depth of dynamic objects from videos. Recent progress in dynamic video depth prediction has focused on improving the output of monocular depth estimators by means of multi-view constraints while imposing little to no restrictions on the deformation of the dynamic parts of the scene. However, the theory of Non-Rigid Structure from Motion prescribes to constrain the deformations for 3D reconstruction. We thus propose a new model that departs significantly from this prior work. The idea is to fit a dynamic point cloud to the video data using Sinkhorn's algorithm to associate the 3D points to 2D pixels and use a differentiable point renderer to ensure the compatibility of the 3D deformations with the measured optical flow. In this manner, our algorithm, called Keypoint Transporter, models the overall deformation of the object within the entire video, so it can constrain the reconstruction correspondingly. Compared to weaker deformation models, this significantly reduces the reconstruction ambiguity and, for dynamic objects, allows Keypoint Transporter to obtain reconstructions of the quality superior or at least comparable to prior approaches while being much faster and reliant on a pre-trained monocular depth estimator network. To assess the method, we evaluate on new datasets of synthetic videos depicting dynamic humans and animals with ground-truth depth. We also show qualitative results on crowd-sourced real-world videos of pets.
A Distributed Threshold Additive Homomorphic Encryption for Federated Learning with Dropout Resiliency Based on Lattice

In federated learning, a parameter server needs to aggregate user gradients and a user needs the privacy of their individual gradients. Among all the possible solutions, additive homomorphic encryption is a natural choice. As users may drop out during a federated learning process, and an adversary could corrupt users and the parameter server, a dropout-resilient scheme with distributed key generation is required. We present a lattice based distributed threshold additive homomorphic encryption scheme with provable security that could be used in the federated learning. The evaluation shows that our proposal has a lower communication overhead among all lattice based proposals when the number of users in FL exceeds 26.
On Phishing: URL Lexical and Network Traffic Features Analysis and Knowledge Extraction using Machine Learning Algorithms (A Comparison Study)

Phishing is a criminal act in which a Phisher creates almost identical website connections exploiting URL Lexical characteristics to dupe unsuspecting users into exposing sensitive information such as financial data, address, and other personal information. Phishers recently sought to trick security experts by masking malicious URLs with obfuscation techniques to make them appear legitimate. This action leads us to conclude that URL lexical features analysis approaches are absolute procedures, and extract analysis techniques are required. This research is a first step toward designing and developing a decision-making system that uses a combination of URL Lexical, and Network Traffic features to detect and classify malicious URLs rather than relying solely on Lexical or URL Network Traffic features. To achieve our goal, we examined and assessed the usage of URL Lexical and Network Traffic features to detect malicious URLs. In the study, three methodologies are used: Complete Features, KMO test as a features selection method, and PCA as a dimensionality method, which are tested by LR, SVM, and KNN classification algorithms and evaluated by the Confusion Matrix Accuracy measure. Using Network Traffics features (ISCXURL dataset), the W/O approach: LR, SVM, and KNN has 92%, 94%, and 93% accuracy. The KMO approach: SVM has 91% accuracy. The PCA approach: LR and SVM have 92% and 94% accuracy, surpassing the use of Lexical features (UCI dataset). In contrast, using Lexical features (UCI dataset), the KMO approach: LR and KNN has 90% and 94% accuracy. The PCA approach: KNN has 95% accuracy, surpassing the use of Network Traffic features (ISCXURL dataset). As a result, we are confident in proceeding with the next step of designing and developing a decision-making application that detects and classifies malicious URLs utilizing URL Lexical and Network Traffic features.
Contextual Outpainting with Object-Level Contrastive Learning

We study the problem of contextual outpainting, which aims to hallucinate the missing background contents based on the remaining foreground contents. Existing image outpainting methods focus on completing object shapes or extending existing scenery textures, neglecting the semantically meaningful relationship between the missing and remaining contents. To explore the semantic cues provided by the remaining foreground contents, we propose a novel ConTextual Outpainting GAN (CTO-GAN), leveraging the semantic layout as a bridge to synthesize coherent and diverse background contents. To model the contextual correlation between foreground and background contents, we incorporate an object-level contrastive loss to regularize the learning of cross-modal representations of foreground contents and the corresponding background semantic layout, facilitating accurate semantic reasoning. Furthermore, we improve the realism of the generated background contents via detecting generated context in adversarial training. Extensive experiments demonstrate that the proposed method achieves superior performance compared with existing solutions on the challenging COCO-stuff dataset. Project page: https://ddlee-cn.github.io/cto-gan.
An OT Forensic Model Based on Established IT Forensics Using IIRA

Through Industrie 4.0, the connectivity of automation or industrial control systems (ICS) is increasing. Simultaneously the frequency and aggressiveness of remote attacks have increased, impacting the risk and vulnerabilities of ICS [1]. For example, a remote attacker can now damage industrial machines or impair the efficiency of production facilities over the Internet. To find evidence and protect oneself against future attacks as well as to minimize risks, it is necessary to be able to investigate or trace down such incidents by performing forensics within ICS. In this paper, we developed a forensic model to plan and perform forensic investigations within Internet-based ICS. For this purpose, we use established classic IT forensic processes as well as existing forensic research for Operational Technology (OT). Our model is compliant with the Industrial Internet Reference Architecture (IIRA) so it can be integrated or applied to existing Internet-based ICS.The approach is validated by showing that it is suitable to detect and analyze a recently published, more sophisticated attack on the OT specific components of typical ICS architectures.
Sensor Fusion Operators for Multimodal 2D Object Detection

Autonomous driving requires effective capabilities to detect road objects in different environmental conditions. One promising solution to improve perception is to leverage multi-sensor fusion. This approach aims to combine various sensor streams in order to best integrate the information coming from the different sensors. Fusion operators are used to combine features from different modalities inside convolutional neural network architectures. In this study, we provide a framework for evaluating early fusion operators using different 2D object detection architectures. This comparative study includes element-wise addition and multiplication, feature concatenation, multi-modal factorized bilinear pooling, and bilaterally-guided fusion. We report quantitative results of the performance as well as an analysis of computational costs of these operators on different architectures.
DetectorDetective: Investigating the Effects of Adversarial Examples on Object Detectors

With deep learning based systems performing exceedingly well in many vision-related tasks, a major concern with their widespread deployment especially in safety-critical applications is their susceptibility to adversarial attacks. We propose DetectorDetective, an interactive visual tool that aims to help users better understand the behaviors of a model as adversarial images journey through an object detector. DetectorDetective enables users to easily learn about how the three key modules of the Faster R-CNN object detector — Feature Pyramidal Network, Region Proposal Network, and Region Of Interest Head — respond to a user-selected benign image and its adversarial version. Visualizations about the progressive changes in the intermediate features among such modules help users gain insights into the impact of adversarial attacks, and perform side-by-side comparisons between the benign and adversarial responses. Furthermore, DetectorDetective displays saliency maps for the input images to comparatively highlight image regions that contribute to attack success. DetectorDetective complements adversarial machine learning research on object detection by providing a user-friendly interactive tool for inspecting and understanding model responses. DetectorDetective is available at the following public demo link: https://poloclub.github.io/detector-detective. A video demo is available at https://youtu.be/5C3Klh87CZI.
Meta-GF: Training Dynamic-Depth Neural Networks Harmoniously

Most state-of-the-art deep neural networks use static inference graphs, which makes it impossible for such networks to dynamically adjust the depth or width of the network according to the complexity of the input data. Different from these static models, depth-adaptive neural networks, e.g. the multi-exit networks, aim at improving the computation efficiency by conducting adaptive inference conditioned on the input. To achieve adaptive inference, multiple output exits are attached at different depths of the multi-exit networks. Unfortunately, these exits usually interfere with each other in the training stage. The interference would reduce performance of the models and cause negative influences on the convergence speed. To address this problem, we investigate the gradient conflict of these multi-exit networks, and propose a novel meta-learning based training paradigm namely Meta-GF (meta gradient fusion) to harmoniously train these exits. Different from existing approaches, Meta-GF takes account of the importances of the shared parameters to each exit, and fuses the gradients of each exit by the meta-learned weights. Experimental results on CIFAR and ImageNet verify the effectiveness of the proposed method. Furthermore, the proposed Meta-GF requires no modification on the network structures and can be directly combined with previous training techniques. The code is available at https://github.com/SYVAE/MetaGF .
Scalable Video Coding for Humans and Machines

Video content is watched not only by humans, but increasingly also by machines. For example, machine learning models analyze surveillance video for security and traffic moni-toring, search through YouTube videos for inappropriate content, and so on. In this paper, we propose a scalable video coding framework that supports machine vision (specifically, object detection) through its base layer bitstream and human vision via its enhancement layer bitstream. The proposed framework includes components from both conventional and Deep Neural Network (DNN)-based video coding. The results show that on object detection, the proposed framework achieves 13–19% bit savings compared to state-of-the-art video codecs, while remaining competitive in terms of MS-SSIM on the human vision task.
Real-time Semantic Segmentation for Aggregating Long-Range Information and Region Understanding

With the improvement of hardware performance, semantic segmentation based on convolutional neural network has achieved wide concern for its' advantage of high accuracy. However, in practical application scenarios, such as autonomous driving, what we care about is not only the segmentation accuracy, but also the ability of the model to process information in real-time on edge devices. In this paper, a lightweight and efficient real-time semantic segmentation network was proposed, which is based on a two-way structure, aggregates detailed spatial information and high-level semantic information, integrates long-range dependencies and region-level understanding. The proposed algorithm achieves good performance. We conducted experiments based on an NVIDIA Geforce RTX 2080 SUPER graphics card and achieved 73.8% accuracy on Cityscapes dataset with a speed of 211 FPS, and 72.8% accuracy on Camvid dataset.
Fast Kötter–Nielsen–Høholdt Interpolation over Skew Polynomial Rings

Skew polynomials are a class of non-commutative polynomials that have several applications in computer science, coding theory and cryptography. In particular, skew polynomials can be used to construct and decode evaluation codes in several metrics, like e.g. the Hamming, rank, sum-rank and skew metric. We propose a fast divide-and-conquer variant of the Kötter-Nielsen-Høholdt (KNH) interpolation algorithm: it inputs a list of linear functionals on skew polynomial vectors, and outputs a reduced Gröbner basis of their kernel intersection. This can be used to solve the interpolation step of interpolation-based decoding of (interleaved) Gabidulin, linearized Reed—Solomon and skew Reed—Solomon codes efficiently, which have various applications in coding theory and code-based quantum-resistant cryptography.
Improved Lightweight Target Detection Algorithm for Complex Roads with YOLOv5

Aiming at the problems of false detection and missed detection caused by dense occluded targets and small targets in complex road background, and the existing target detection algorithms are not strong enough, a complex road lightweight target detection algorithm based on improved YOLOv5 is proposed. Firstly, Quality Focal Loss is introduced, and the classification score is combined with the quality prediction of location to improve the positioning accuracy of dense occluded targets; Secondly, a shallow detection layer is added as the detection layer of smaller targets, the three-scale detection of the original algorithm is changed to four scales, and the feature fusion part is also changed accordingly, which improves the learning ability of the algorithm to the features of small targets; Then, referring to the lightweight idea of GhostNet, the Bottleneck of the Neck part of the original algorithm is replaced by GhostBottleneck, which reduces the volume of the algorithm model and the amount of parameters and calculations on the premise of maintaining the detection accuracy; Finally, the Backbone part of the original algorithm is replaced by Mobilenetv3, which further lightens the algorithm and improves the detection speed of the algorithm in the CPU environment. The experimental results show that on the public autopilot data set KITTI and the self-made rider helmet data set Helmet, compared with the original YOLOv5s algorithm, the volume of the improved algorithm in this paper is reduced by 60%, the amount of parameters is reduced by 62%, and the amount of calculation is reduced by 63%. The detection speed in the CPU environment is increased by 6FPS. Compared with some mainstream target detection algorithms, the improved algorithm in this paper has certain advantages.
Towards a Framework for Adapting Machine Learning Components

Machine Learning (ML) models are now commonly used as components in systems. As any other component, ML components can produce erroneous outputs that may penalize system utility. In this context, self-adaptive systems emerge as a natural approach to cope with ML mispredictions, through the execution of adaptation tactics such as model retraining. To synthesize an adaptation strategy, the self-adaptation manager needs to reason about the cost-benefit tradeoffs of the applicable tactics, which is a non-trivial task for tactics such as model retraining, whose benefits are both context- and data-dependent.To address this challenge, this paper proposes a probabilistic modeling framework that supports automated reasoning about the cost/benefit tradeoffs associated with improving ML components of ML-based systems. The key idea of the proposed approach is to decouple the problems of (i) estimating the expected performance improvement after retrain and (ii) estimating the impact of ML improved predictions on overall system utility.We demonstrate the application of the proposed framework by using it to self-adapt a state-of-the-art ML-based fraud-detection system, which we evaluate using a publicly-available, real fraud detection dataset. We show that by predicting system utility stemming from retraining a ML component, the probabilistic model checker can generate adaptation strategies that are significantly closer to the optimal, as compared against baselines such as periodic retraining, or reactive retraining.
Mutually Reinforcing Structure with Proposal Contrastive Consistency for Few-Shot Object Detection

Few-shot object detection is based on the base set with abundant labeled samples to detect novel categories with scarce samples. The majority of former solutions are mainly based on meta-learning or transfer-learning, neglecting the fact that images from the base set might contain unlabeled novel-class objects, which easily leads to performance degradation and poor plasticity since those novel objects are served as the background. Based on the above phenomena, we propose a Mutually Reinforcing Structure Network (MRSN) to make rational use of unlabeled novel class instances in the base set. In particular, MRSN consists of a mining model which unearths unlabeled novel-class instances and an absorbed model which learns variable knowledge. Then, we design a Proposal Contrastive Consistency (PCC) module in the absorbed model to fully exploit class characteristics and avoid bias from unearthed labels. Furthermore,we propose a simple and effective data synthesis method undirectional-CutMix (UD-CutMix) to improve the robustness of model mining novel class instances, urge the model to pay attention to discriminative parts of objects and eliminate the interference of background information. Extensive experiments illustrate that our proposed approach achieves state-of-the-art results on PASCAL VOC and MS-COCO datasets. Our code will be released at https://github.com/MMatx/MRSN .
AgeTransGAN for Facial Age Transformation with Rectified Performance Metrics

We propose the AgeTransGAN for facial age transformation and the improvements to the metrics for performance evaluation. The AgeTransGAN is composed of an encoder-decoder generator and a conditional multitask discriminator with an age classifier embedded. The generator considers cycle-generation consistency, age classification and cross-age identity consistency to disentangle the identity and age characteristics during training. The discriminator fuses age features with the target age group label and collaborates with the embedded age classifier to warrant the desired target age generation. As many previous work use the Face++ APIs as the metrics for performance evaluation, we reveal via experiments the inappropriateness of using the Face++ as the metrics for the face verification and age estimation of juniors. To rectify the Face++ metrics, we made the Cross-Age Face (CAF) dataset which contains 4000 face images of 520 individuals taken from their childhood to seniorhood. The CAF is one of the very few datasets that offer far more images of the same individuals across large age gaps than the popular FG-Net. We use the CAF to rectify the face verification thresholds of the Face++ APIs across different age gaps. We also use the CAF and FFHQ-Aging datasets to compare the age estimation performance of the Face++ APIs and an age estimator that we made, and propose rectified metrics for performance evaluation. We compare the AgeTransGAN with state-of-the-art approaches by using the existing and rectified metrics.