Reviewerly/exHarmony huggingface dataset:
Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels

While neural networks trained for semantic segmentation are essential for perception in autonomous driving, most current algorithms assume a fixed number of classes, presenting a major limitation when developing new autonomous driving systems with the need of additional classes. In this paper we present a technique implementing class-incremental learning for semantic segmentation without using the labeled data the model was initially trained on. Previous approaches still either rely on labels for both old and new classes, or fail to properly distinguish between them. We show how to overcome these problems with a novel class-incremental learning technique, which nonetheless requires labels only for the new classes. Specifically, (i) we introduce a new loss function that neither relies on old data nor on old labels, (ii) we show how new classes can be integrated in a modular fashion into pretrained semantic segmentation models, and finally (iii) we re-implement previous approaches in a unified setting to compare them to ours. We evaluate our method on the Cityscapes dataset, where we exceed the mIoU performance of all baselines by 3.5% absolute reaching a result, which is only 2.2% absolute below the upper performance limit of single-stage training, relying on all data and labels simultaneously.
Unbiased Learning to Rank in Feeds Recommendation

In feeds recommendation, users are able to constantly browse items generated by never-ending feeds using mobile phones. The implicit feedback from users is an important resource for learning to rank, however, building ranking functions from such observed data is recognized to be biased. The presentation of the items will influence the user's judgements and therefore introduces biases. Most previous works in the unbiased learning to rank literature focus on position bias (i.e., an item ranked higher has more chances of being examined and interacted with). By analyzing user behaviors in product feeds recommendation, in this paper, we identify and introduce context bias, which refers to the probability that a user interacting with an item is biased by its surroundings, to unbiased learning to rank. We propose an Unbiased Learning to Rank with Combinational Propensity (ULTR-CP) framework to remove the inherent biases jointly caused by multiple factors. Under this framework, a context-aware position bias model is instantiated to estimate the unified bias considering both position and context biases. In addition to evaluating propensity score estimation approaches by the ranking metrics, we also discuss the evaluation of the propensities directly by checking their balancing properties. Extensive experiments performed on a real e-commerce data set collected from JD.com verify the effectiveness of context bias and illustrate the superiority of ULTR-CP against the state-of-the-art methods.
Prediction, Estimation, and Attribution

Summary The scientific needs and computational limitations of the twentieth century fashioned classical statistical methodology. Both the needs and limitations have changed in the twenty‐first, and so has the methodology. Large‐scale prediction algorithms—neural nets, deep learning, boosting, support vector machines, random forests—have achieved star status in the popular press. They are recognizable as heirs to the regression tradition, but ones carried out at enormous scale and on titanic datasets. How do these algorithms compare with standard regression techniques such as ordinary least squares or logistic regression? Several key discrepancies will be examined, centering on the differences between prediction and estimation or prediction and attribution (significance testing). Most of the discussion is carried out through small numerical examples.
Detecting anomalies in time series data from a manufacturing system using recurrent neural networks

The industrial internet of things allows manufacturers to acquire large amounts of data. This opportunity, assuming the right methods are available, allows manufacturers to find anomalies that arise during manufacturing system operation. Data acquired from a manufacturing system are usually in the forms of time series. This paper proposes a new method that can detect anomalies in time series data. This model is based on recurrent neural networks, and it can be trained using data acquired during routine system operation. This is very beneficial because often, there are few data labeled as anomalies, since anomalies are hopefully rare events in a well-managed manufacturing system. The model takes time series data as an input and reconstructs the input data. Time series data with an anomaly would causes patterns in the reconstruction errors that are inconsistent with error patterns of anomaly-free data. The performance of the proposed method is assessed using data from a diesel engine assembly process. Three common types of anomalies are detected from the time series data. It is shown that the method not only can detect anomalies, but it can also provide insights into the timestep at which the anomaly occurred. This feature helps a manufacturer pinpoint the source of the problem.
Skedulix: Hybrid Cloud Scheduling for Cost-Efficient Execution of Serverless Applications

We present a framework for scheduling multifunction serverless applications over a hybrid public-private cloud. A set of serverless jobs is input as a batch, and the objective is to schedule function executions over the hybrid platform to minimize the cost of public cloud use, while completing all jobs by a specified deadline. As this scheduling problem is NP-Hard, we propose a greedy algorithm that dynamically determines both the order and placement of each function execution using predictive models of function execution time and network latencies. We present a prototype implementation of our framework that uses AWS Lambda and OpenFaaS, for the public and private cloud, respectively. We evaluate our prototype in live experiments using a mixture of compute and I/O heavy serverless applications. Our results show that our framework can achieve a speedup in batch processing of up to 1.92 times that of an approach that uses only the private cloud, at 40.5% the cost of an approach that uses only the public cloud.
AI System Engineering—Key Challenges and Lessons Learned

The main challenges are discussed together with the lessons learned from past and ongoing research along the development cycle of machine learning systems. This will be done by taking into account intrinsic conditions of nowadays deep learning models, data and software quality issues and human-centered artificial intelligence (AI) postulates, including confidentiality and ethical aspects. The analysis outlines a fundamental theory-practice gap which superimposes the challenges of AI system engineering at the level of data quality assurance, model building, software engineering and deployment. The aim of this paper is to pinpoint research topics to explore approaches to address these challenges.
Efficient Federated Learning for Cloud-Based AIoT Applications

As a promising method for central model training on decentralized device data without compromising user privacy, federated learning (FL) is becoming more and more popular in Internet-of-Things (IoT) design. However, due to limited computing and memory resources of devices that restrict the capabilities of hosted deep learning models, existing FL approaches for artificial intelligence IoT (AIoT) applications suffer from inaccurate prediction results. To address this problem, this article presents a collaborative <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Big.Little</i> branch architecture to enable efficient FL for AIoT applications. Inspired by the architecture of BranchyNet which has multiple prediction branches, our approach deploys deep neural network (DNN) models across both cloud and AIoT devices. Our <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Big.Little</i> branch model has two branches, where the big branch is deployed on cloud for strengthened prediction accuracy, and the little branches are used to fit for AIoT devices. When AIoT devices cannot make the prediction with high confidence using local little branches, they will resort to the big branch for further inference. To increase both prediction accuracy and early exit rate of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Big.Little</i> branch model, we propose a two-stage training and coinference scheme, which considers the local characteristics of AIoT scenarios. Comprehensive experiment results obtained from a real AIoT environment demonstrate the efficiency and effectiveness of our approach in terms of prediction accuracy and average inference time.
SSHC: A Secure and Scalable Hybrid Consensus Protocol for Sharding Blockchains With a Formal Security Framework

Sharding blockchains are proposed to solve the scalability problem while maintaining security and decentralization. However, there are still many issues to be solved. First, the member selection and assignment process are not strictly analyzed, which might lead to an increase in the adversary proportion. Second, current intra-shard consensus algorithms are inefficient. Besides, cross-shard transaction processing costs expensive system overhead. Moreover, there is a lack of a formal security framework. In this article, we propose a secure and scalable hybrid consensus (SSHC). First, we propose a fair sharding selection scheme to select committee members, including mining processes and member lists confirmation by a reference committee. Second, a pipelined Byzantine fault tolerance for intra-shard consensus is designed, combining the pipelined technology with threshold signatures. Third, we propose a responsive sharding transaction batch processing mechanism to handle cross-shard transactions, which reduces the number of calls to Byzantine fault tolerance algorithms. Fourth, a secure committee reconfiguration method is designed to update shard members efficiently. Furthermore, we employ a formal security framework to design and analyze a sharding blockchain. For an adversary whose computational power fraction is less than <inline-formula><tex-math notation="LaTeX">$1/3$</tex-math></inline-formula> , by reasonably setting a corruption parameter and other related parameters, SSHC is proved to achieve consistency and liveness.
Multi-feature fusion for specific emitter identification via deep ensemble learning

Specific emitter identification (SEI) is an important problem in the field of electronic intelligence. There are two major limitations in most existing SEI methods: First, the features should be artificially extracted, which requires specialized expertise; Second, various features are not merged effectively to improve performance. In this paper, an automatic multi-feature extraction and fusion method based on deep ensemble learning is proposed for SEI. This method extracts and fuses multiple features via data-driven strategies using convolutional neural networks (CNN). No additional constraints are required on the type and number of the features to be fused. Three typical characteristics of radar pulse signals, including amplitude, phase and spectrum asymmetry, are taken as example features for method description and performance verification. Experiment results demonstrate that, the proposed method performs well in automatic feature extraction, and it can significantly improve the SEI performance via multi-feature fusion.
Single nighttime image dehazing based on image decomposition

Dehazing plays an important role in promoting the performance of outdoor computer vision systems. However, existing dehazing methods are targeted to daytime haze scenes, and are not able to improve visual effects for nighttime hazy images due to the unpredictable factors at night. In this paper, an effective single image dehazing framework based on image decomposition is presented for nighttime hazy images. First, the input single nighttime image is separated into the glow-shaped image and the glow-free nighttime hazy image using its relative smoothness constraint. Then, a novel structure-texture-noise decomposition model based on the exponentiated mean local variance is devised to split the nighttime hazy image into a structure layer, a texture layer and a noise layer, in which the structure layer and the texture layer are dehazed based on the maximum reflectance prior and the dark channel prior and enhanced in the gradient domain respectively. Finally, the dehazed structure layer and the enhanced texture layer are fused to produce a dehazed result. Experiments demonstrate that the proposed approach outperforms several state-of-the-art dehazing techniques for nighttime hazy scenes, especially in terms of noise suppression. Besides, the proposed algorithm is also capable of handling daytime hazy images and low-light degraded images.
DualSR: Zero-Shot Dual Learning for Real-World Super-Resolution

Advanced methods for single image super-resolution (SISR) based upon Deep learning have demonstrated a remarkable reconstruction performance on downscaled images. However, for real-world low-resolution images (e.g. images captured straight from the camera) they often generate blurry images and highlight unpleasant artifacts. The main reason is the training data that does not reflect the real-world super-resolution problem. They train the net-work using images downsampled with an ideal (usually bicubic) kernel. However, for real-world images the degradation process is more complex and can vary from image to image. This paper proposes a new dual-path architecture (DualSR) that learns an image-specific low-to-high resolution mapping using only patches of the input test image. For every image, a downsampler learns the degradation process using a generative adversarial network, and an up-sampler learns to super-resolve that specific image. In the DualSR architecture, the upsampler and downsampler are trained simultaneously and they improve each other using cycle consistency losses. For better visual quality and eliminating undesired artifacts, the upsampler is constrained by a masked interpolation loss. On standard benchmarks with unknown degradation kernels, DualSR outperforms recent blind and non-blind super-resolution methods in term of SSIM and generates images with higher perceptual quality. On real-world LR images it generates visually pleasing and artifact-free results.
Efficient and reliable forensics using intelligent edge computing

Due to the increasing awareness and use of cloud and edge computing, society and industries are beginning to understand the benefits they can provide. Cloud and Edge are the future of information management, and they have transformed the Internet into an innovative and interactive computing platform. The ultimate goal of edge/cloud computing is to reduce the use of computing resources in the network, as well as support information sharing and intercommunication efforts within the network. Secure edge computing methodologies are applied in both open and heterogeneous network systems to protect them from many potential security threats. However, these approaches only provide passive protection for normal edge computing operations, and fail to address the security measures of several applications, particularly forensics in industrial settings. Forensics applications running on edge computing must be capable of support taking legal action against invaders for malicious damage or information theft. This paper proposes an efficient and reliable forensics framework (ERFF) to address industrial intelligent edge computing critical for the industry 4.0 implementation plan. The proposed ERFF consists of a detective module and validation model, with the detective module responsible for detecting the interaction between the client terminal and the edge resource, which means the investigator is capable of gathering the evidence securely. The security-validation model integrated with ERFF is far safer than sharing common key-based cryptographic approaches. The proposed conceptual framework is tested with Live Digital Forensic Framework for a Cloud (LDF2C), and results are compared with other existing industrial frameworks that fulfill fundamental ISO/IEC 17025 accreditation requirements, including Legal Reliable Forensic Framework (LRFF), Source Identification Network Forensics Framework (SINFF) and Logging Framework for Cloud Computing Forensic (LFCCF)). These frameworks were designed to support the digital forensic requirements of industry and academia, and experimental results validate the effectiveness of the proposed framework from reliability and efficiency perspectives as well as realistic scenarios
Centralized Cooperative Sensor Fusion for Dynamic Sensor Network With Limited Field-of-View via Labeled Multi-Bernoulli Filter

This paper presents a new solution for multi-target tracking over a network of sensors with limited spatial coverage. The proposed solution is based on the centralized data fusion architecture. The main contribution of the paper is the introduction of a new track-to-track fusion approach in which the posterior distributions of multi-target states, reported by various sensor nodes, are fused in a way that the redundant information are combined and the rest complement each other. The proposed solution is formulated within the labeled random finite set framework in which the fused posterior incorporates all the state and label information provided by multiple sensor nodes. The performance of the proposed method is evaluated via simulation experiments that involve challenging tracking scenarios. The proposed method is implemented using sequential Monte Carlo method and the results confirm its effectiveness.
Region-Based Dehazing via Dual-Supervised Triple-Convolutional Network

Most physical model-based dehazing methods are subject to contrast degradation in a dark or shadow region because of the mismatch between the physical model and real haze. This degradation decreases the quality of dehazed images. Furthermore, the retinex-haze combined models can cause the brightness saturation problem in a haze region. For this reason, the retinex-haze combined approaches are not appropriate to enhance the real-world haze images. To solve these problems, we present a novel region-based dehazing method via dual-supervised triple-convolutional network (TCN). More specifically, the proposed network first simulates the mismatch problem based on the region-model. Next, we then train the proposed triple-convolutional network, which can estimate the degraded regions. We then present a novel dual-supervised learning method to efficiently train the networks using a non-ideal dataset. Experimental results show that the proposed method outperforms state-of-the-art approaches in solving complex haze problems. The output of the proposed network has a high-similarity index in most cases for various benchmark dataset. Our approach also produces high-quality images in real haze image datasets.
Blockchain-Based Decentralized Public Auditing for Cloud Storage

Public auditing schemes for cloud storage systems have been extensively explored with the increasing importance of data integrity. A third-party auditor (TPA) is introduced in public auditing schemes to verify the integrity of outsourced data on behalf of users. To resist malicious TPAs, many blockchain-based public verification schemes have been proposed. However, existing auditing schemes rely on a centralized TPA, and they are vulnerable to tempting auditors who may collude with malicious blockchain miners to produce biased auditing results. In this article, we propose a blockchain-based decentralized public auditing (BDPA) scheme by utilizing a decentralized blockchain network to undertake the responsibility of a centralized TPA, and also mitigate the influence of tempting auditors and malicious blockchain miners by taking the concept of decentralized autonomous organization (DAO). A detailed security analysis shows that BDPA can preserve data integrity against tempting auditors and malicious blockchain miners. A comprehensive performance evaluation demonstrates that BDPA is feasible and scalable.
A posteriori corrections to the iterative qubit coupled cluster method to minimize the use of quantum resources in large-scale calculations

Abstract The iterative qubit coupled cluster (iQCC) method is a systematic variational approach to solve the electronic structure problem on universal quantum computers. It is able to use arbitrarily shallow quantum circuits at expense of iterative canonical transformation of the Hamiltonian and rebuilding a circuit. Here we present a variety of a posteriori corrections to the iQCC energies to reduce the number of iterations to achieve the desired accuracy. Our energy corrections are based on a low-order perturbation theory series that can be efficiently evaluated on a classical computer. Moreover, capturing a part of the total energy perturbatively, allows us to formulate the qubit active-space concept, in which only a subset of all qubits is treated variationally. As a result, further reduction of quantum resource requirements is achieved. We demonstrate the utility and efficiency of our approach numerically on the examples of 10-qubit N 2 molecule dissociation, the 24-qubit H 2 O symmetric stretch, and 56-qubit singlet-triplet gap calculations for the technologically important complex, tris-(2-phenylpyridine)iridium(III) Ir(ppy) 3 .
Foreground-aware Semantic Representations for Image Harmonization

Image harmonization is an important step in photo editing to achieve visual consistency in composite images by adjusting the appearances of a foreground to make it compatible with a background. Previous approaches to harmonize composites are based on training of encoder-decoder networks from scratch, which makes it challenging for a neural network to learn a high-level representation of objects. We propose a novel architecture to utilize the space of high-level features learned by a pre-trained classification network. We create our models as a combination of existing encoder-decoder architectures and a pretrained foreground-aware deep high-resolution network. We extensively evaluate the proposed method on the existing image harmonization benchmark and set up a new state-of-the-art in terms of MSE and PSNR metrics. The code and trained models are available publicly.