Reviewerly/exHarmony huggingface dataset:
Sparse, collaborative, or nonnegative representation: Which helps pattern classification?

The use of sparse representation (SR) and collaborative representation (CR) for pattern classification has been widely studied in tasks such as face recognition and object categorization. Despite the success of SR/CR based classifiers, it is still arguable whether it is the ℓ1-norm sparsity or the ℓ2-norm collaborative property that brings the success of SR/CR based classification. In this paper, we investigate the use of nonnegative representation (NR) for pattern classification, which is largely ignored by previous work. Our analyses reveal that NR can boost the representation power of homogeneous samples while limiting the representation power of heterogeneous samples, making the representation sparse and discriminative simultaneously and thus providing a more effective solution to representation based classification than SR/CR. Our experiments demonstrate that the proposed NR based classifier (NRC) outperforms previous representation based classifiers. With deep features as inputs, it also achieves state-of-the-art performance on various visual classification tasks.
PDNet: Prior-Model Guided Depth-Enhanced Network for Salient Object Detection

Fully convolutional neural networks (FCNs) have shown outstanding performance in many computer vision tasks including salient object detection. However, there still remains two issues needed to be addressed in deep learning based saliency detection. One is the lack of tremendous amount of annotated data to train a network. The other is the lack of robustness for extracting salient objects in images containing complex scenes. In this paper, we present a new architecture-PDNet, a robust prior-model guided depth-enhanced network for RGB-D salient object detection. In contrast to existing works, in which RGB-D values of image pixels are fed directly to a network, the proposed architecture is composed of a master network for processing RGB values, and a sub-network making full use of depth cues and incorporate depth-based features into the master network. To overcome the limited size of the labeled RGB-D dataset for training, we employ a large conventional RGB dataset to pre-train the master network, which proves to contribute largely to the final accuracy. Extensive evaluations over five benchmark datasets demonstrate that our proposed method performs favorably against the state-of-the-art approaches.
explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning

We propose a framework for interactive and explainable machine learning that enables users to (1) understand machine learning models; (2) diagnose model limitations using different explainable AI methods; as well as (3) refine and optimize the models. Our framework combines an iterative XAI pipeline with eight global monitoring and steering mechanisms, including quality monitoring, provenance tracking, model comparison, and trust building. To operationalize the framework, we present explAIner, a visual analytics system for interactive and explainable machine learning that instantiates all phases of the suggested pipeline within the commonly used TensorBoard environment. We performed a user-study with nine participants across different expertise levels to examine their perception of our workflow and to collect suggestions to fill the gap between our system and framework. The evaluation confirms that our tightly integrated system leads to an informed machine learning process while disclosing opportunities for further extensions.
No Blind Spots: Full-Surround Multi-Object Tracking for Autonomous Vehicles Using Cameras and LiDARs

Online multi-object tracking (MOT) is extremely important for high-level spatial reasoning and path planning for autonomous and highly-automated vehicles. In this paper, we present a modular framework for tracking multiple objects (vehicles), capable of accepting object proposals from different sensor modalities (vision and range) and a variable number of sensors, to produce continuous object tracks. This work is a generalization of the MDP framework for MOT proposed by Xiang et al., with some key extensions - First, we track objects across multiple cameras and across different sensor modalities. This is done by fusing object proposals across sensors accurately and efficiently. Second, the objects of interest (targets) are tracked directly in the real world. This is a departure from traditional techniques where objects are simply tracked in the image plane. Doing so allows the tracks to be readily used by an autonomous agent for navigation and related tasks. To verify the effectiveness of our approach, we test it on real world highway data collected from a heavily sensorized testbed capable of capturing full-surround information. We demonstrate that our framework is well-suited to track objects through entire maneuvers around the ego-vehicle, some of which take more than a few minutes to complete. We also leverage the modularity of our approach by comparing the effects of including/excluding different sensors, changing the total number of sensors, and the quality of object proposals on the final tracking result.
Estimating surface solar irradiance from satellites: Past, present, and future perspectives

Surface Solar Irradiance (SSI) is a key parameter dictating surface-atmosphere interactions, driving radiative, hydrological, and land surface processes, and can thus impinge greatly upon weather and climate. It is thereby a prerequisite of many studies and applications. Estimating SSI from satellites began in the 1960s, and is currently the principal way to map SSI spatiotemporal distributions from regional to global scales. Starting from an overview of historical studies carried out in the past several decades, this paper reviews the progresses made in methodology, validation, and products over these years. First, the requirements of SSI in various studies or applications are presented along with the theoretical background of SSI satellite estimation. Methods to estimate SSI from satellites are then summarized as well as their advantages and limitations. Validations of satellite-based SSI on two typical spatial scales are discussed followed by a brief description of existing products and their accuracies. Finally, the challenges faced by current SSI satellite estimation are analyzed, and possible improvements to implement in the future are suggested. This review not only updates the review paper by Pinker et al. (1995) on satellite methods to derive SSI but also offers a more comprehensive summary of the related studies and applications.
Kappa Updated Ensemble for drifting data stream mining

Learning from data streams in the presence of concept drift is among the biggest challenges of contemporary machine learning. Algorithms designed for such scenarios must take into an account the potentially unbounded size of data, its constantly changing nature, and the requirement for real-time processing. Ensemble approaches for data stream mining have gained significant popularity, due to their high predictive capabilities and effective mechanisms for alleviating concept drift. In this paper, we propose a new ensemble method named Kappa Updated Ensemble (KUE). It is a combination of online and block-based ensemble approaches that uses Kappa statistic for dynamic weighting and selection of base classifiers. In order to achieve a higher diversity among base learners, each of them is trained using a different subset of features and updated with new instances with given probability following a Poisson distribution. Furthermore, we update the ensemble with new classifiers only when they contribute positively to the improvement of the quality of the ensemble. Finally, each base classifier in KUE is capable of abstaining itself for taking a part in voting, thus increasing the overall robustness of KUE. An extensive experimental study shows that KUE is capable of outperforming state-of-the-art ensembles on standard and imbalanced drifting data streams while having a low computational complexity. Moreover, we analyze the use of Kappa versus accuracy to drive the criterion to select and update the classifiers, the contribution of the abstaining mechanism, the contribution of the diversification of classifiers, and the contribution of the hybrid architecture to update the classifiers in an online manner.
Analog Coupled Oscillator Based Weighted Ising Machine

We report on an analog computing system with coupled non-linear oscillators which is capable of solving complex combinatorial optimization problems using the weighted Ising model. The circuit is composed of a fully-connected 4-node LC oscillator network with low-cost electronic components and compatible with traditional integrated circuit technologies. We present the theoretical modeling, experimental characterization, and statistical analysis our system, demonstrating single-run ground state accuracies of 98% on randomized MAX-CUT problem sets with binary weights and 84% with 5-bit weight resolutions. Solutions are obtained within 5 oscillator cycles, and the time-to-solution has been demonstrated to scale directly with oscillator frequency. We present scaling analysis which suggests that large coupled oscillator networks may be used to solve computationally intensive problems faster and more efficiently than conventional algorithms. The proof-of-concept system presented here provides the foundation for realizing such larger scale systems using existing hardware technologies and could pave the way towards an entirely novel computing paradigm.
Learn to Scale: Generating Multipolar Normalized Density Maps for Crowd Counting

Dense crowd counting aims to predict thousands of human instances from an image, by calculating integrals of a density map over image pixels. Existing approaches mainly suffer from the extreme density variations. Such density pattern shift poses challenges even for multi-scale model ensembling. In this paper, we propose a simple yet effective approach to tackle this problem. First, a patch-level density map is extracted by a density estimation model and further grouped into several density levels which are determined over full datasets. Second, each patch density map is automatically normalized by an online center learning strategy with a multipolar center loss. Such a design can significantly condense the density distribution into several clusters, and enable that the density variance can be learned by a single model. Extensive experiments demonstrate the superiority of the proposed method. Our work outperforms the state-of-the-art by 4.2%, 14.3%, 27.1% and 20.1% in MAE, on the ShanghaiTech Part A, ShanghaiTech Part B, UCF_CC_50 and UCF-QNRF datasets, respectively.
Free-Form Video Inpainting With 3D Gated Convolution and Temporal PatchGAN

Free-form video inpainting is a very challenging task that could be widely used for video editing such as text removal (see Fig. 1). Existing patch-based methods could not handle non-repetitive structures such as faces, while directly applying image-based inpainting models to videos will result in temporal inconsistency (see videos). In this paper, we introduce a deep learning based free-form video inpainting model, with proposed 3D gated convolutions to tackle the uncertainty offree-form masks and a novel Temporal PatchGAN loss to enhance temporal consistency. In addition, we collect videos and design a free-form mask generation algorithm to build the free-form video inpainting (FVI) dataset for training and evaluation of video inpainting models. We demonstrate the benefits of these components and experiments on both the FaceForensics and our FVI dataset suggest that our method is superior to existing ones. Related source code, full-resolution result videos and the FVI dataset could be found on Github.
FLChain: A Blockchain for Auditable Federated Learning with Trust and Incentive

Federated learning (shorted as FL) recently proposed by Google is a privacy-preserving method to integrate distributed data trainers. FL is extremely useful due to its ensuring privacy, lower latency, less power consumption and smarter models, but it could fail if multiple trainers abort training or send malformed messages to its partners. Such misbehavior are not auditable and parameter server may compute incorrectly due to single point failure. Furthermore, FL has no incentive to attract sufficient distributed training data and computation power. In this paper, we propose FLChain to build a decentralized, public auditable and healthy FL ecosystem with trust and incentive. FLChain replace traditional FL parameter server whose computation result must be consensual on-chain. Our work is not trivial when it is vital and hard to provide enough incentive and deterrence to distributed trainers. We achieve model commercialization by providing a healthy marketplace for collaborative-training models. Honest trainer can gain fairly partitioned profit from well-trained model according to its contribution and the malicious can be timely detected and heavily punished. To reduce the time cost of misbehavior detecting and model query, we design DDCBF for accelerating the query of blockchain-documented information. Finally, we implement a prototype of our work and measure the cost of various operations.
The impact of blockchain technology on business models – a taxonomy and archetypal patterns

Abstract Blockchain technology enables new ways of organizing economic activities, reduces costs and time associated with intermediaries, and strengthens the trust in an ecosystem of actors. The impact of this seminal technology is reflected by an upcoming research stream and various firms that examine the potential uses of blockchain technology. While there are promising use cases of this new technology, research and practice are still in their infancy about altering existing and creating new business models. We develop a taxonomy of blockchain business models based on 99 blockchain ventures to explore the impact of blockchain technology on business models. As a result, we identify five archetypal patterns, which enhance our understanding of how blockchain technology affects existing and creates new business models. We propose to use these results to discover further patterns fueled by blockchain technology and illustrate how firms can use blockchain technology to innovate their business models.
Image Description With Polar Harmonic Fourier Moments

Due to their good rotational invariance and stability, image continuous orthogonal moments are intensively applied in rotationally invariant recognition and image processing. However, most moments produce numerical instability, which impacts the image reconstruction and recognition performance. In this paper, a new set of invariant continuous orthogonal moments, polar harmonic Fourier moments (PHFMs), free of numerical instability is designed. The radial basis functions (RBFs) of the PHFMs are much simpler than those of the Chebyshev-Fourier moments (CHFMs), orthogonal Fourier-Mellin moments (OFMMs), Zernike moments (ZMs), and pseudo-Zernike moments (PZMs). For the same degree, the RBFs of the PHFMs have more zeros and are more evenly distributed than those of the ZMs and PZMs. Therefore, PHFMs do not suffer from information suppression problem; hence, the image description ability of the PHFMs is superior to that of the ZMs and PZMs. Moreover, the RBFs of the PHFMs are always less than or equal to 1.0 near the unit disk center, whereas those of the OFMMs, PZMs, CHFMs, and radial harmonic Fourier moments (RHFMs) are infinite (implying numerical instability). This indicates that PHFMs can outperform these moments in image reconstruction tasks. We theoretically and experimentally demonstrate that PHFMs outperform the above moments in reconstructing images and recognizing rotationally invariant objects considering noise and various attacks. This paper also details the significance of the PHFM phase in image reconstruction, angle estimation using PHFMs, and the accurate moment selection of the PHFMs.
Amplitude estimation without phase estimation

This paper focuses on the quantum amplitude estimation algorithm, which is a core subroutine in quantum computation for various applications. The conventional approach for amplitude estimation is to use the phase estimation algorithm, which consists of many controlled amplification operations followed by a quantum Fourier transform. However, the whole procedure is hard to implement with current and near-term quantum computers. In this paper, we propose a quantum amplitude estimation algorithm without the use of expensive controlled operations; the key idea is to utilize the maximum likelihood estimation based on the combined measurement data produced from quantum circuits with different numbers of amplitude amplification operations. Numerical simulations we conducted demonstrate that our algorithm asymptotically achieves nearly the optimal quantum speedup with a reasonable circuit length.
The need for privacy with public digital contact tracing during the COVID-19 pandemic

Digital contact tracing applications represent a powerful yet controversial strategy to combat the COVID-19 pandemic. Manual contact tracing has important challenges, not limited to recall bias and delays in communicating with high-risk contacts.1 Digital technologies are already increasingly used in the context of health-care delivery and clinical trials.2 Due to the considerable strain on public health institutions, digital contact tracing through mobile phones is being used or explored in a growing number of countries despite concerns raised over individual privacy and state surveillance.
Single-Path NAS: Designing Hardware-Efficient ConvNets in Less Than 4 Hours

Can we automatically design a Convolutional Network (ConvNet) with the highest image classification accuracy under the latency constraint of a mobile device? Neural architecture search (NAS) has revolutionized the design of hardware-efficient ConvNets by automating this process. However, the NAS problem remains challenging due to the combinatorially large design space, causing a significant searching time (at least 200 GPU-hours). To alleviate this complexity, we propose Single-Path NAS, a novel differentiable NAS method for designing hardware-efficient ConvNets in less than 4 h. Our contributions are as follows: 1. Single-path search space: Compared to previous differentiable NAS methods, Single-Path NAS uses one single-path over-parameterized ConvNet to encode all architectural decisions with shared convolutional kernel parameters, hence drastically decreasing the number of trainable parameters and the search cost down to few epochs. 2. Hardware-efficient ImageNet classification: Single-Path NAS achieves $$74.96\%$$ top-1 accuracy on ImageNet with 79 ms latency on a Pixel 1 phone, which is state-of-the-art accuracy compared to NAS methods with similar inference latency constraints ($$\le $$80 ms). 3. NAS efficiency: Single-Path NAS search cost is only 8 epochs (30 TPU-hours), which is up to 5,000$$\times $$ faster compared to prior work. 4. Reproducibility: Unlike all recent mobile-efficient NAS methods which only release pretrained models, we open-source our entire codebase at: https://github.com/dstamoulis/single-path-nas .
Learning Meta Face Recognition in Unseen Domains

Face recognition systems are usually faced with unseen domains in real-world applications and show unsatisfactory performance due to their poor generalization. For example, a well-trained model on webface data cannot deal with the ID vs. Spot task in surveillance scenario. In this paper, we aim to learn a generalized model that can directly handle new unseen domains without any model updating. To this end, we propose a novel face recognition method via meta-learning named Meta Face Recognition (MFR). MFR synthesizes the source/target domain shift with a meta-optimization objective, which requires the model to learn effective representations not only on synthesized source domains but also on synthesized target domains. Specifically, we build domain-shift batches through a domain-level sampling strategy and get back-propagated gradients/meta-gradients on synthesized source/target domains by optimizing multi-domain distributions. The gradients and meta-gradients are further combined to update the model to improve generalization. Besides, we propose two benchmarks for generalized face recognition evaluation. Experiments on our benchmarks validate the generalization of our method compared to several baselines and other state-of-the-arts. The proposed benchmarks and code will be available at https://github.com/cleardusk/MFR.
Local-Global Video-Text Interactions for Temporal Grounding

This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query. We tackle this problem using a novel regression-based model that learns to extract a collection of mid-level features for semantic phrases in a text query, which corresponds to important semantic entities described in the query (e.g., actors, objects, and actions), and reflect bi-modal interactions between the linguistic features of the query and the visual features of the video in multiple levels. The proposed method effectively predicts the target time interval by exploiting contextual information from local to global during bi-modal interactions. Through in-depth ablation studies, we find out that incorporating both local and global context in video and text interactions is crucial to the accurate grounding. Our experiment shows that the proposed method outperforms the state of the arts on Charades-STA and ActivityNet Captions datasets by large margins, 7.44% and 4.61% points at Recall@tIoU=0.5 metric, respectively.
High-Resolution Vehicle Trajectory Extraction and Denoising From Aerial Videos

In recent years, unmanned aerial vehicle (UAV) has become an increasingly popular tool for traffic monitoring and data collection on highways due to its advantage of low cost, high resolution, good flexibility, and wide spatial coverage. Extracting high-resolution vehicle trajectory data from aerial videos taken by a UAV flying over target highway segment becomes a critical research task for traffic flow modeling and analysis. This study aims at proposing a novel methodological framework for automatic and accurate vehicle trajectory extraction from aerial videos. The method starts by developing an ensemble detector to detect vehicles in the target region. Then, the kernelized correlation filter is applied to track vehicles fast and accurately. After that, a mapping algorithm is proposed to transform vehicle positions from the Cartesian coordinates in image to the Frenet coordinates to extract raw vehicle trajectories along the roadway curves. The data denoising is then performed using a wavelet transform to eliminate the biased vehicle trajectory positions. Our method is tested on two aerial videos taken on different urban expressway segments in both peak and non-peak hours on weekdays. The extracted vehicle trajectories are compared with manual calibrated data to testify the framework performance. The experimental results show that the proposed method successfully extracts vehicle trajectories with a high accuracy: the measurement error of Mean Squared Deviation is 2.301 m, the Root-mean-square deviation is 0.175 m, and the Pearson correlation coefficient is 0.999. The video and trajectory data in this study are publicly accessible for serving as benchmark at https://seutraffic.com.