Reviewerly/exHarmony huggingface dataset:
Beyond the Visible Pixels Using Semantic Amodal Segmentation in Remote Sensing Images

2D representations of 3D scenes generate occlusions among different targets. Understanding targets by only seeing parts of them is referred to as an amodal perception, which is still unexplored in remote sensing. Thus, we propose integrating this concept using peculiarities of remote sensing Nadir images to classify non-visible targets at a pixel level. Nadir images present a hierarchical order of occlusions, allowing us to separate different layers. We developed a dataset with 600 images and three classes (roads, vehicles, and trees) with in-dependent labelling for each class. Any semantic segmentation model is suitable for this task, but we explored the U-net architecture with three backbones (Efficient-net-B7, ResNet-101, and ResNeXt-101). The evaluation considered the IoU metric, providing 80% for the best model (Efficient-net-B7). Future studies aim to extend this approach by introducing competing classes among each layer and increasing the number of samples and categories.
Planning under periodic observations: bounds and bounding-based solutions

We study planning problems faced by robots operating in uncertain environments with incomplete knowledge of state, and actions that are noisy and/or imprecise. This paper identifies a new problem sub-class that models settings in which information is revealed only intermittently through some exogenous process that provides state information periodically. Several practical domains fit this model, including the specific scenario that motivates our research: autonomous navigation of a planetary exploration rover augmented by remote imaging. With an eye to efficient specialized solution methods, we examine the structure of instances of this sub-class. They lead to Markov Decision Processes with exponentially large action-spaces but for which, as those actions comprise sequences of more atomic elements, one may establish performance bounds by comparing policies under different information assumptions. This provides a way in which to construct performance bounds systematically. Such bounds are useful because, in conjunction with the insights they confer, they can be employed in bounding-based methods to obtain high-quality solutions efficiently; the empirical results we present demonstrate their effectiveness for the considered problems. The foregoing has also alluded to the distinctive role that time plays for these problems -more specifically: time until information is revealed- and we uncover and discuss several interesting subtleties in this regard.
Can Eye Tracking with Pervasive Webcams Replace Dedicated Eye Trackers? an Experimental Comparison of Eye-Tracking Performance

Eye tracking (ET) is increasingly used in cognitive science and human-computer interaction research. Currently, however, expensive ET devices are required, which limits their pervasive application, for example, as a tool for digital learning. The emergence of low-cost Artificial Intelligence (AI)-based consumer-grade webcam eye tracking, partially already available as open-source software, promises to change this situation. It is currently unclear (1) what performance in terms of, e.g., tracking accuracy, calibration stability and real time sampling rate stability can be achieved with webcam ET (wcET) in comparison to using expensive dedicated hardware eye tracking (dhET); and (2) how the expected performance degradation affects ET-based applications. In this work, we address the first question and present a wcET system, which we developed based on open source code and publicly available data sets. We ran this system with a consumer-grade Logitech BRIO Ultra HD Pro webcam that provides a stream of HD images 60 Hz simultaneously to a commercial remote eye tracker, the Tobii Pro X3-120, which delivers tracked gaze points at a rate of 120 Hz. Based on recordings of 20 participants (age: 27.0±4.5 years), we assessed the data quality in terms of accuracy, precision, and sampling rate stability. The observed performance of the wcET system (accuracy: 2.5 $$^{\circ }\pm 0.7^{\circ }$$ , precision: 0.3 $$^{\circ }\pm 0.3^{\circ }$$ ) is, as expected, worse than the performance of the dhET system (accuracy: 0.9 $$^{\circ }\pm 0.9^{\circ }$$ , precision: 0.7 $$^{\circ }\pm 0.8^{\circ }$$ ), which it is sufficient for many applications, especially if the stimulus can be designed and adapted to the gaze tracking quality correspondingly. Running the wcET system in real-time, we obtained a sampling rate of 26.3±1.03 Hz, i.e., nearly the frequency with which the camera provides images.
Collaborative Scalable Visual Compression for Human-Centered Videos

Machine intelligence systems have been increasingly widely deployed in real-world circumstances, while the conventional human-vision oriented video coding schemes are inefficient to be embedded in large-scale systems and further support a wide range of applications. There have been urgent demands for a new generation of compression framework to efficiently encodes visual data, where the compression and analytics for machine vision and human perception can be jointly optimized. To this end, we propose a novel visual compression framework to provide visual contents with different granularity for both human and machine vision tasks collaboratively. The proposed scalable compression framework maintains the critical semantic information in a basic layer, so that it is capable of supporting the accurate machine vision analysis under a tight bit-rate constraint. It is scalable to provide visual representations of different granularity to support various kinds of tasks, including video reconstruction that serves human vision examination. Experimental results on the human-centered videos have demonstrated the promising functionality of scalable visual coding with improved efficiency for high-performance machine analysis and human perception.
Optimization Model of Digital Higher Education - SMART University 4.0

The new industrial revolution, based on digitalization and networking, which we are witnessing, requires modernization of most economic institutions, including the education system, and offers for this modernization not only a new set of priorities, technologies, but also new principles of flexibility, individualization, maximization of efficiency. In other words, it creates an opportunity and a need, necessary and sufficient conditions for building a new model of education. The main tasks in creating a new e-university are the introduction of new processes of design, production, systematization, storage and delivery of educational content, as well as modernization of management and administration of the educational process.This article discusses a set of proposals for building a new optimized and adaptive model for higher education, based on the concept of digital SMART - University or University 4.0.
A Heuristic for an Online Applicability of Anomaly Detection Techniques

OHODIN is an online extension for data streams of the kNN-based ODIN anomaly detection approach. It provides a detection-threshold heuristic that is based on extreme value theory. In contrast to sophisticated anomaly and novelty detection approaches the decision-making process of ODIN is interpretable by humans, making it interesting for certain applications. However, it is limited in terms of the underlying detection method. In this article, we present an extension of the OHODIN to further detection techniques to reinforce OHODIN capability of online data streams anomaly detection. We introduce the algorithm modifications and an experimental evaluation with competing state-of-the-art anomaly detection approaches.
FAFMOTS: A Fast and Anchor Free Method for Online Joint Multi-Object Tracking and Segmentation

Fast object tracking system is important to Augmented Reality (AR). However, tracking multiple objects fast and accurately is not a trivial task in computer vision, especially when the objects become crowded and frequently occluded. In this paper, we design an efficient segmentation model along with the tracking and carefully study the contributions of pixel-level appearance modeling in the presence of occlusions. Different from previous joint tracking and segmentation methods that depend on bounding boxes for instance segmentation, we proposed an anchor free model to speed the object mask estimation. Specifically, our proposed approach jointly addresses instance segmentation and tracking with a single convolutional network for the first time, and generates object masks to explicitly separate overlapping objects in tracking. As pointed out by several previous works, the goals of tracking and segmentation are not always consistent, we deliberately design three homogeneous branches in our network to balance the competitive three functional parts of objectness, masks, and re-identification simultaneously and constrain them in a comparable scale. With the effective three functional parts, our method could run fast for the time-sensitive tracking task and meanwhile relieve the effects of appearance noise brought by the occlusions. Experimental results on the MOTS benchmark demonstrate the effectiveness of the proposed method.
Learn to Adapt to New Environments From Past Experience and Few Pilot Blocks

In recent years, deep learning has been widely applied in communications and achieved remarkable performance improvement. Most of the existing works are based on data-driven deep learning, which requires a significant amount of training data for the communication model to adapt to new environments and results in huge computing resources for collecting data and retraining the model. In this paper, we will significantly reduce the required amount of training data for new environments by leveraging the learning experience from the known environments. Therefore, we introduce few-shot learning to enable the communication model to generalize to new environments, which is realized by an attention-based method. With the attention network embedded into the deep learning-based communication model, environments with different power delay profiles can be learnt together in the training process, which is called the learning experience. By exploiting the learning experience, the communication model only requires few pilot blocks to perform well in the new environment. Through an example of deep-learning-based channel estimation, we demonstrate that this novel design method achieves better performance than the existing data-driven approach designed for few-shot learning.
Validating Event-B models using PDDL

The formal Event-B method makes it possible to obtain systems or software correct by construction. The modeling activity aims to establish formal models in Event-B. As for the proof activity, it aims at correcting these models. A proven Event-B model is not necessarily valid. We propose to open the Event-B formal method on PDDL in order to further validate Event-B models. We aim to validate the dynamic properties related to the sequence of events authorized on Event-B models. PDDL is a de facto standard language for planning problems. It is equipped with planners to automatically generate solution-plans. To achieve this, we successfully used the MDE approach, Xtext to create an integrated development environment specific to the Event-B language and the Xtend tool to implement our transformation EventB2PDDL and produce PDDL descriptions using a top-down approach.
Data Discernment for Affordable Training in Medical Image Segmentation

Collecting sufficient high-quality training data for deep neural networks is often expensive or even unaffordable in medical image segmentation tasks. We thus propose to train the network by using external data that can be collected in a cheaper way, e.g., crowd-sourcing. We show that by data discernment, the network is able to mine valuable knowledge from external data, even though the data distribution is very different from that of the original (internal) data. We discern the external data by learning an importance weight for each of them, with the goal to enhance the contribution of informative external data to network updating, while suppressing the data that are 'useless' or even 'harmful'. An iterative algorithm that alternatively estimates the importance weight and updates the network is developed by formulating the data discernment as a constrained nonlinear programming problem. It estimates the importance weight according to the distribution discrepancy between the external data and the internal dataset, and imposes a constraint to drive the network to learn more effectively, compared with the network without using the external data. We evaluate the proposed algorithm on two tasks: abdominal CT image and cervical smear image segmentation, using totally 6 publicly available datasets. The effectiveness of the algorithm is demonstrated by extensive experiments. Source codes are available at: https://github.com/YouyiSong/Data-Discernment.
Cover Reproducible Steganography via Deep Generative Models

Whereas cryptography easily arouses attacks by means of encrypting a secret message into a suspicious form, steganography is advantageous for its resilience to attacks by concealing the message in an innocent-looking cover signal. Minimal distortion steganography, one of the mainstream steganography frameworks, embeds messages while minimizing the distortion caused by the modification on the cover elements. Due to the unavailability of the original cover signal for the receiver, message embedding is realized by finding the coset leader of the syndrome function of steganographic codes migrated from channel coding, which is complex and has limited performance. Fortunately, deep generative models and the robust semantic of generated data make it possible for the receiver to perfectly reproduce the cover signal from the stego signal. With this advantage, we propose cover-reproducible steganography where the source coding, e.g., arithmetic coding, serves as the steganographic code. Specifically, the decoding process of arithmetic coding is used for message embedding and its encoding process is regarded as message extraction. Taking text-to-speech and text-to-image synthesis tasks as two examples, we illustrate the feasibility of cover-reproducible steganography. Steganalysis experiments and theoretical analysis are conducted to demonstrate that the proposed methods outperform the existing methods in most cases.
An Improved Monte Carlo Denoising Algorithm Based on Kernel-Predicting Convolutional Network

Monte Carlo(MC) path tracing renders images with severe noise under one-sample-per-pixel conditions, and one of the challenges of denoising is to achieve high rendering quality with low time overhead. The Kernel prediction method is a high-quality denoising algorithm. In this paper, we propose an improved kernel prediction network algorithm to remove Monte Carlo noise with interactive rate. During training, we construct a variant of the multi-scale ResBlock structure to improve the denoising ability of our network. During deployment, we remove the residual connection by using a reserving and merging operation on ResBlock to equivalently transform multi-branch topology to a single-branch model. Experiments show that compared with previous methods, our method achieves a better trade-off between time expenditure and denoising quality. When removing Monte Carlo noise, we can improve the denoising quality without introducing high time overhead.
PHOcus

Our ability to collect data is rapidly outstripping our ability to effectively store and use it. Organizations are therefore facing tough decisions of what data to archive (or dispose of) to effectively meet their business goals. PHOcus addresses this problem in the context of image data (photos) by proposing which photos to archive to meet an online storage budget. The decision is based on factors such as usage patterns and their relative importance, the quality and size of a photo, the relevance of a photo for a usage pattern, the similarity between different photos, as well as policy requirements of what photos must be retained. We formalize the photo archival problem and give an efficient algorithm with an optimal approximation guarantee. We then demonstrate our system, PHOcus, on an e-commerce application as well as with personal photos on a smartphone, and discuss how many of the inputs to the problem can be automatically obtained.
Principal Components of Neural Convolution Filters

Convolutions in neural networks are still essential on various vision tasks. To develop neural convolutions, this study focuses on Structured Receptive Field (SRF), representing a convolution filter as a linear combination of widely acting designed components. Although SRF can represent convolution filters with fewer components than the number of filter bins, N-Jet, the sole component system implementation, requires ten trainable parameters per filter to improve accuracy even for 3 × 3 convolutions. Hence, we aim to formulate a new component system for SRF that can represent valid filters with fewer components. Our component system named “OtX” is based on the Principal Component Analysis of well-trained filter weights because the extracted components will also be principal for neural convolution filters. In addition to proposing the component system, we develop a component scaling method to defuse massive scale differences among the coefficients in a linear combination of OtX components. In the experimental section, we train image classification models on CIFAR-100 dataset under the hyperparameters tuned for the original models with the standard convolutions. For NFNet-F0 classifier, OtX with six components performs 0.5% better than the standard convolution, 3.1% better than N-Jet with six components, and only 0.1% worse than N-Jet with ten components. Besides, OtX with nine components provides stabler training than N-Jet, performing 0.5% better than the standard for NFNet-F0. OtX suits when replacing standard convolutions because OtX performs at least comparably against N-Jet with further parameter efficiency and training stability.
An Improved Lion Swarm Optimization Algorithm With Chaotic Mutation Strategy and Boundary Mutation Strategy for Global Optimization

Lion swarm optimization (LSO) inspired by the natural division of labor among lion king, lionesses and lion cubs in a lion group, i.e., lion king guarding, lionesses hunting and lion cubs following, is a relatively novel swarm intelligent optimization technique.Due to its remarkable performance, the canonical LSO has been extensively researched.However, how to balance contradictions between the exploration and the exploitation and alleviate the premature convergence are two critical concerns that need to be dealt with in the LSO study.To address these two drawbacks, enhance the optimization performance, and broaden its application domain, an improved lion swarm optimization algorithm with chaotic mutation strategy and a boundary mutation strategy (CBLSO) is proposed in this paper.In the proposed algorithm, a chaotic mutation strategy based on chaotic cubic mapping is designed to enhance the exploration ability of the algorithm, while the boundary mutation strategy based on the concept of multilevel parallel is adopted to manage boundary constraint violations, which is beneficial for improving the exploitation ability of the algorithm.The proposed CBLSO is evaluated on 56 classic test functions and 30 CEC2014 benchmark functions, and is compared with quite a few state-of-the-art algorithms regarding often-used performance metrics.The experimental results demonstrate the superior performance of the embedded strategies on balancing the exploration and the exploitation.Furthermore, the proposed CBLSO is applied to the optimal dispatch problem of cascade hydropower stations based on a novel constraints handling method designed in this paper to validate its good practicability and performance.The experimental results of a case study on the optimal dispatch problem of China's Wujiang cascade hydropower stations indicate that the proposed CBLSO can produce better and more reliable optimal results than the canonical LSO and other comparison algorithms with competitive speed.Thus, we can conclude that the proposed CBLSO is a competitive and effective alterative tool to solve complex numerical optimization problems and real-world optimization with complicated constraints.
Training Quantised Neural Networks with STE Variants: the Additive Noise Annealing Algorithm

Training quantised neural networks (QNNs) is a non-differentiable optimisation problem since weights and features are output by piecewise constant functions. The standard solution is to apply the straight-through estimator (STE), using different functions during the inference and gradient computation steps. Several STE variants have been proposed in the literature aiming to maximise the task accuracy of the trained network. In this paper, we analyse STE variants and study their impact on QNN training. We first observe that most such variants can be modelled as stochastic regularisations of stair functions; although this intuitive interpretation is not new, our rigorous discussion generalises to further variants. Then, we analyse QNNs mixing different regularisations, finding that some suitably synchronised smoothing of each layer map is required to guarantee pointwise compositional convergence to the target discontinuous function. Based on these theoretical insights, we propose additive noise annealing (ANA), a new algorithm to train QNNs encompassing standard STE and its variants as special cases. When testing ANA on the CIFAR-J0 image classification benchmark, we find that the major impact on task accuracy is not due to the qualitative shape of the regularisations but to the proper synchronisation of the different STE variants used in a network, in accordance with the theoretical results.
Explaining Learning Performance with Local Performance Regions and Maximally Relevant Meta-Rules

Identifying instances in a learning task that are difficult to predict is important to avoid critical errors at deployment time. Additionally, providing explanations for good or bad predictions of a model can be useful to understand its behavior and to plan how to improve it (e.g., by data augmentation in specific areas of instances). In this paper, we propose a method to provide explanations for a model's predictive performance based on the induction of meta-rules. Each meta-rule identifies a local region in the instance space, called Local Performance Region (LPR). The meta-rules are induced using a reduced number of attributes, in such a way that each LPR can be inspected by, e.g., plotting a pairwise attribute plot. Additionally, given a group of instances to explain (or eventually an individual instance), we propose a greedy-search algorithm that finds the subset of non-redundant LPRs that maximally covers the instances. By explaining the (in)correctness of model predictions, LPRs constitute a novel use of meta-learning and a novel application in explainable AI. Experiments show the usefulness of LPRs while explaining inaccurate class predictions of Random Forest in a benchmark dataset, demonstrating a special case of LPRs, called Local Hard Regions (LHRs).