Reviewerly/exHarmony huggingface dataset:
Refined UNet v3: Efficient end-to-end patch-wise network for cloud and shadow segmentation with multi-channel spectral features

Semantic segmentation is one of the essential prerequisites for computer vision tasks, but edge-precise segmentation stays challenging due to the potential lack of a proper model indicating the low-level relation between pixels. We have presented Refined UNet v2, a concatenation of a network backbone and a subsequent embedded conditional random field (CRF) layer, which coarsely performs pixel-wise classification and refines edges of segmentation regions in a one-stage way. However, the CRF layer of v2 employs a gray-scale global observation (image) to construct contrast-sensitive bilateral features, which is not able to achieve the desired performance on ambiguous edges. In addition, the naïve depth-wise Gaussian filter cannot always compute efficiently, especially for a longer-range message-passing step. To address the aforementioned issues, we upgrade the bilateral message-passing kernel and the efficient implementation of Gaussian filtering in the CRF layer in this paper, referred to as Refined UNet v3, which is able to effectively capture ambiguous edges and accelerate the message-passing procedure. Specifically, the inherited UNet is employed to coarsely locate cloud and shadow regions and the embedded CRF layer refines the edges of the forthcoming segmentation proposals. The multi-channel guided Gaussian filter is applied to the bilateral message-passing step, which improves detecting ambiguous edges that are hard for the gray-scale counterpart to identify, and fast Fourier transform-based (FFT-based) Gaussian filtering facilitates an efficient and potentially range-agnostic implementation. Furthermore, Refined UNet v3 is able to be extended to segmentation on multi-spectral datasets, and the corresponding refinement examination confirms the development of shadow retrieval. Experiments and corresponding results demonstrate that the proposed update can outperform its counterpart in terms of the detection of vague edges, shadow retrieval, and isolated redundant regions, and it is practically efficient in our TensorFlow implementation. The demo source code is available at https://github.com/92xianshen/refined-unet-v3.
Image Generation for 2D-CNN Using Time-Series Signal Features from Foot Gesture Applied to Select Cobot Operating Mode

Advances in robotics are part of reducing the burden associated with manufacturing tasks in workers. For example, the cobot could be used as a “third-arm” during the assembling task. Thus, the necessity of designing new intuitive control modalities arises. This paper presents a foot gesture approach centered on robot control constraints to switch between four operating modalities. This control scheme is based on raw data acquired by an instrumented insole located at a human’s foot. It is composed of an inertial measurement unit (IMU) and four force sensors. Firstly, a gesture dictionary was proposed and, from data acquired, a set of 78 features was computed with a statistical approach, and later reduced to 3 via variance analysis ANOVA. Then, the time series collected data were converted into a 2D image and provided as an input for a 2D convolutional neural network (CNN) for the recognition of foot gestures. Every gesture was assimilated to a predefined cobot operating mode. The offline recognition rate appears to be highly dependent on the features to be considered and their spatial representation in 2D image. We achieve a higher recognition rate for a specific representation of features by sets of triangular and rectangular forms. These results were encouraging in the use of CNN to recognize foot gestures, which then will be associated with a command to control an industrial robot.
IPPODAMO

The steady deployment of IoT is paving the road toward concrete implementations of the smart city concept, allowing public/private institutions to sense and model (near) real-time digital replicas of physical processes and environments. This Digital Twin (DT) could be used proactively, as a decision support system, providing insights into possible optimizations of processes in a smart city context.
Feature Weighted Non-Negative Matrix Factorization

Non-negative matrix factorization (NMF) is one of the most popular techniques for data representation and clustering and has been widely used in machine learning and data analysis. NMF concentrates the features of each sample into a vector and approximates it by the linear combination of basis vectors, such that the low-dimensional representations are achieved. However, in real-world applications, the features usually have different importance. To exploit the discriminative features, some methods project the samples into the subspace with a transformation matrix, which disturbs the original feature attributes and neglects the diversity of samples. To alleviate the above problems, we propose the feature weighted NMF (FNMF) in this article. The salient properties of FNMF can be summarized as three-fold: 1) it learns the weights of features adaptively according to their importance; 2) it utilizes multiple feature weighting components to preserve the diversity; and 3) it can be solved efficiently with the suggested optimization algorithm. The performance on synthetic and real-world datasets demonstrates that the proposed method obtains the state-of-the-art performance.
Bermudan option pricing by quantum amplitude estimation and Chebyshev interpolation

Abstract Pricing of financial derivatives, in particular early exercisable options such as Bermudan options, is an important but heavy numerical task in financial institutions, and its speed-up will provide a large business impact. Recently, applications of quantum computing to financial problems have been started to be investigated. In this paper, we first propose a quantum algorithm for Bermudan option pricing. This method performs the approximation of the continuation value, which is a crucial part of Bermudan option pricing, by Chebyshev interpolation, using the values at interpolation nodes estimated by quantum amplitude estimation. In this method, the number of calls to the oracle to generate underlying asset price paths scales as $\widetilde{O}(\epsilon ^{-1})$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mover> <mml:mi>O</mml:mi> <mml:mo>˜</mml:mo> </mml:mover> <mml:mo>(</mml:mo> <mml:msup> <mml:mi>ϵ</mml:mi> <mml:mrow> <mml:mo>−</mml:mo> <mml:mn>1</mml:mn> </mml:mrow> </mml:msup> <mml:mo>)</mml:mo> </mml:math> , where ϵ is the error tolerance of the option price. This means the quadratic speed-up compared with classical Monte Carlo-based methods such as least-squares Monte Carlo, in which the oracle call number is $\widetilde{O}(\epsilon ^{-2})$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mover> <mml:mi>O</mml:mi> <mml:mo>˜</mml:mo> </mml:mover> <mml:mo>(</mml:mo> <mml:msup> <mml:mi>ϵ</mml:mi> <mml:mrow> <mml:mo>−</mml:mo> <mml:mn>2</mml:mn> </mml:mrow> </mml:msup> <mml:mo>)</mml:mo> </mml:math> .
Deep Neural Fuzzy System Oriented toward High-Dimensional Data and Interpretable Artificial Intelligence

Fuzzy systems (FSs) are popular and interpretable machine learning methods, represented by the adaptive neuro-fuzzy inference system (ANFIS). However, they have difficulty dealing with high-dimensional data due to the curse of dimensionality. To effectively handle high-dimensional data and ensure optimal performance, this paper presents a deep neural fuzzy system (DNFS) based on the subtractive clustering-based ANFIS (SC-ANFIS). Inspired by deep learning, the SC-ANFIS is proposed and adopted as a submodule to construct the DNFS in a bottom-up way. Through the ensemble learning and hierarchical learning of submodules, DNFS can not only achieve faster convergence, but also complete the computation in a reasonable time with high accuracy and interpretability. By adjusting the deep structure and the parameters of the DNFS, the performance can be improved further. This paper also performed a profound study of the structure and the combination of the submodule inputs for the DNFS. Experimental results on five regression datasets with various dimensionality demonstrated that the proposed DNFS can not only solve the curse of dimensionality, but also achieve higher accuracy, less complexity, and better interpretability than previous FSs. The superiority of the DNFS is also validated over other recent algorithms especially when the dimensionality of the data is higher. Furthermore, the DNFS built with five inputs for each submodule and two inputs shared between adjacent submodules had the best performance. The performance of the DNFS can be improved by distributing the features with high correlation with the output to each submodule. Given the results of the current study, it is expected that the DNFS will be used to solve general high-dimensional regression problems efficiently with high accuracy and better interpretability.
Applying Machine Learning in Fog Computing Environments for Panoramic Teeth Imaging

This article provides a brief description of the functions of X-rays for dental treatment. The new possibilities of computer technology for the presentation of panoramic images for a dentist are described. A new way of interaction of a doctor with sources of X-ray images using the Internet of Things is proposed. A description of the architecture is given according to three standards at the design stage. Smartphone application has been developed. The effectiveness of the use of fog computing environments and cloud technologies for recognizing tooth damage using machine learning has been proven. Specific examples are given.
Towards the detection of driver–pedestrian eye contact

Non-verbal communication, such as eye contact between drivers and pedestrians, has been regarded as one way to reduce accident risk. So far, studies have assumed rather than objectively measured the occurrence of eye contact. We address this research gap by developing an eye contact detection method and testing it in an indoor experiment with scripted driver–pedestrian interactions at a pedestrian crossing. Thirty participants acted as a pedestrian either standing on an imaginary curb or crossing an imaginary one-lane road in front of a stationary vehicle with an experimenter in the driver's seat. In half of the trials, pedestrians were instructed to make eye contact with the driver; in the other half, they were prohibited from doing so. Both parties' gaze was recorded using eye trackers. An in-vehicle stereo camera recorded the car's point of view, a head-mounted camera recorded the pedestrian's point of view, and the location of the driver's and pedestrian's eyes was estimated using image recognition. We demonstrate that eye contact can be detected by measuring the angles between the vector joining the estimated location of the driver's and pedestrian's eyes, and the pedestrian's and driver's instantaneous gaze directions, respectively, and identifying whether these angles fall below a threshold of 4°. We achieved 100% correct classification of the trials involving eye contact and those without eye contact, based on measured eye contact duration. The proposed eye contact detection method may be useful for future research into eye contact.
DIFFBASE: a differential factbase for effective software evolution management

Numerous tools and techniques have been developed to extract and analyze information from software development artifacts. Yet, there is a lack of effective method to process, store, and exchange information among different analyses. In this paper, we propose differential factbase, a uniform exchangeable representation supporting efficient querying and manipulation, based on the existing concept of program facts. We consider program changes as first-class objects, which establish links between intra-version facts of single program snapshots and provide insights on how certain artifacts evolve over time via inter-version facts. We implement a series of differential fact extractors supporting different programming languages and platforms, and demonstrate with usage scenarios the benefits of adopting differential facts in supporting software evolution management.
ZkRep: A Privacy-Preserving Scheme for Reputation-Based Blockchain System

Reputation/Trust-based blockchain systems have attracted considerable research interests for better integrating Internet-of-Things with blockchain in terms of throughput, scalability, energy efficiency and incentive aspects.However, most existing works only consider static adversaries.Hence, they are vulnerable to slowly adaptive attackers, who can target validators with high reputation value to severely degrade the system performance.Therefore, we introduce zkRep, a privacypreserving scheme tailored for reputation-based blockchains.Our basic idea is to hide both the identity and reputation of the validators by periodically changing the identity and reputation commitments (i.e., aliases), which makes it much more difficult for slowly adaptive attackers to identify validators with high reputation value.To realize this idea, we utilize privacypreserving Pedersen-commitment-based reputation updating and leader election schemes that operate on concealed reputations within an epoch.We also introduce a privacy-preserving identity update protocol that changes the identity and time-windowbased cumulative reputation commitments during each epoch transition.We have implemented and evaluated zkRep on the Amazon Web Service.The experimental results and analysis show that zkRep achieves great privacy-preserving features against slowly adaptive attacks with little overhead.
Part-Based Feature Squeezing To Detect Adversarial Examples in Person Re-Identification Networks

Although deep neural networks (DNNs) have achieved top performances in different computer vision tasks, such as object detection, image segmentation and person re-identification (ReID), they can easily be deceived by adversarial examples, which are carefully crafted images with perturbations that are imperceptible to human eyes. Such adversarial examples can significantly degrade the performance of existing DNNs. There are also targeted attacks misleading classifiers into making specific decisions based on attackers’ intentions. In this paper, we propose a new method to effectively detect adversarial examples presented to a person ReID network. The proposed method utilizes parts-based feature squeezing to detect the adversarial examples. We apply two types of squeezing to segmented body parts to better detect adversarial examples. We perform extensive experiments over three major datasets with different attacks, and compare the detection performance of the proposed body part-based approach with a ReID method that is not parts-based. Experimental results show that the proposed method can effectively detect the adversarial examples, and has the potential to avoid significant decreases in person ReID performance caused by adversarial examples.
Guidance And Teaching Network For Video Salient Object Detection

Owing to the difficulties of mining spatial-temporal cues, the existing approaches for video salient object detection (VSOD) are limited in understanding complex and noisy scenarios, and often fail in inferring prominent objects. To alleviate such shortcomings, we propose a simple yet efficient architecture, termed Guidance and Teaching Network (GTNet), to independently distill effective spatial and temporal cues with implicit guidance and explicit teaching at feature- and decision-level, respectively. To be specific, we (a) introduce a temporal modulator to implicitly bridge features from motion into appearance branch, which is capable of fusing cross-modal features collaboratively, and (b) utilise motion-guided mask to propagate the explicit cues during the feature aggregation. This novel learning strategy achieves satisfactory results via decoupling the complex spatial-temporal cues and mapping informative cues across different modalities. Extensive experiments on three challenging benchmarks show that the proposed method can run at$\sim$28 fps on a single TITAN Xp GPU and perform competitively against 14 cutting-edge baselines.
A multi-channel approach for automatic microseismic event association using RANSAC-based Arrival Time Event Clustering (RATEC)

In the presence of background noise, arrival times picked from a surface microseismic data set usually include a number of false picks that can lead to uncertainty in location estimation. To eliminate false picks and improve the accuracy of location estimates, we develop an association algorithm termed RANSAC-based Arrival Time Event Clustering (RATEC) that clusters picked arrival times into event groups based on random sampling and fitting moveout curves that approximate hyperbolas. Arrival times far from the fitted hyperbolas are classified as false picks and removed from the data set prior to location estimation. Simulations of synthetic data for a 1-D linear array show that RATEC is robust under different noise conditions and generally applicable to various types of subsurface structures. By generalizing the underlying moveout model, RATEC is extended to the case of a 2-D surface monitoring array. The effectiveness of event location for the 2-D case is demonstrated using a data set collected by the 5200-element dense Long Beach array. The obtained results suggest that RATEC is effective in removing false picks and hence can be used for phase association before location estimates.
Handwritten Text Generation via Disentangled Representations

Automatically generating handwritten text images is a challenging task due to the diverse handwriting styles and the irregular writing in natural scenes. In this paper, we propose an effective generative model called HTG-GAN to synthesize handwritten text images from latent prior. Unlike single-character synthesis, our method is capable of generating images of sequence characters with arbitrary length, which pays more attention to the structural relationship between characters. We model the structural relationship as the style representation to avoid explicitly modeling the stroke layout. Specifically, the text image is disentangled into style representation and content representation, where the style representation is mapped into Gaussian distribution and the content representation is embedded using character index. In this way, our model can generate new handwritten text images with specified contents and various styles to perform data augmentation, thereby boosting handwritten text recognition (HTR). Experimental results show that our method achieves state-of-the-art performance in handwritten text generation.
M2GUDA

Cross-modal hashing is a critical but very challenging task that is to retrieve similar samples of one modality via queries of other modalities. To improve the unsupervised cross-modal hashing, domain adaptation techniques can be used to support unsupervised hashing learning by transferring semantic knowledge from labeled source domain to unlabeled target domain. However, there are two problems that cannot be ignored: (1) most of domain adaptation based researches mainly focused on unimodal hashing or cross-modal real value-based retrieval but the study for cross-modal hashing is limited; (2) most existing studies only consider one or two consistency constraints during the domain adaptation learning. To this end, this paper propose a novel end-to-end framework to realize unsupervised domain adaptation for cross-modal hashing. This method, dubbed M$^2$GUDA, including four different consistency constraints: structure consistency, domain consistency, semantic consistency and modality consistency for domain adaptation learning. Besides, to enhance the structure consistency learning, we develop a multi-metrics graph modeling method to capture structure information comprehensively. Extensive experiments are performed on three common used benchmarks to evaluate the effectivity of our method. The results show that our method outperforms several state-of-the-art cross-modal hashing methods.
Leveraging EfficientNet and Contrastive Learning for Accurate Global-scale Location Estimation

In this paper, we address the problem of global-scale image geolocation, proposing a mixed classification-retrieval scheme. Unlike other methods that strictly tackle the problem as a classification or retrieval task, we combine the two practices in a unified solution leveraging the advantages of each approach with two different modules. The first leverages the EfficientNet architecture to assign images to a specific geographic cell in a robust way. The second introduces a new residual architecture that is trained with contrastive learning to map input images to an embedding space that minimizes the pairwise geodesic distance of same-location images. For the final location estimation, the two modules are combined with a search-within-cell scheme, where the locations of most similar images from the predicted geographic cell are aggregated based on a spatial clustering scheme. Our approach demonstrates very competitive performance on four public datasets, achieving new state-of-the-art performance in fine granularity scales, i.e., 15.0% at 1km range on Im2GPS3k.
Optimizing Federated Learning on Device Heterogeneity with A Sampling Strategy

Federated learning (FL) is a novel machine learning that performs distributed training locally on devices and aggregating the local models into a global one. The limited network bandwidth and the tremendous amount of model data that need to be transported bring up expensive communication cost. Meanwhile, heterogeneity in the devices’ local datasets and computation power exerts a huge influence on the performance of FL. To address these issues, we provide an empirical and mathematical analysis of device heterogeneity on the performance of model convergence and quality, then propose a holistic design to efficiently sample devices. Furthermore, we design a dynamic strategy to further speed up convergence and propose the FedAgg algorithm to alleviate the deviation caused by device heterogeneity. With extensive experiments performed in PyTorch, we show that the number of communication rounds required in FL can be reduced by up to 52% on the MNIST dataset, 32% on CIFAR-10, and 28% on FashionMNIST in comparison to the Federated Averaging algorithm.
Summarized distributions of mass: a statistical approach to consumers’ consumption spaces

This paper focuses on logical aspects of choices being made by the consumer under conditions of uncertainty or certainty. Such logical aspects are found out to be the same. Choices being made by the consumer that should maximize her subjective utility are decisions studied by revealed preference theory. A finite number of possible alternatives is considered. They are mutually exclusive propositions identifying all quantitative states of nature of a consumption plan. Each proposition of it is expressed by a real number. This research work distinguishes it from its temporary truth value depending on the state of information and knowledge of the consumer. Since each point of the consumption space of the consumer belongs to a two-dimensional convex set, this article focuses on conjoint distributions of mass. Indeed, the consumption space of the consumer is generated by all coherent summaries of a conjoint distribution of mass. Each point of her consumption space is connected with a weighted average of states of nature of two consumption plans jointly studied. They give rise to a conjoint distribution of mass. The consumer chooses a point of a two-dimensional convex set representing that bundle of goods actually demanded by her inside of her consumption space. This paper innovatively shows that it is nothing but a bilinear and disaggregate measure. It is decomposed into two real numbers, where each real number is a linear measure. In this paper, different measures are obtained. They can be disaggregate or aggregate measures, where the latter are independent of the notion of ordered pair of consumption plans.