Reviewerly/exHarmony huggingface dataset:
Seesaw Loss for Long-Tailed Instance Segmentation

Instance segmentation has witnessed a remarkable progress on class-balanced benchmarks. However, they fail to perform as accurately in real-world scenarios, where the category distribution of objects naturally comes with a long tail. Instances of head classes dominate a long-tailed dataset and they serve as negative samples of tail categories. The overwhelming gradients of negative samples on tail classes lead to a biased learning process for classifiers. Consequently, objects of tail categories are more likely to be misclassified as backgrounds or head categories. To tackle this problem, we propose Seesaw Loss to dynamically re-balance gradients of positive and negative samples for each category, with two complementary factors, i.e., mitigation factor and compensation factor. The mitigation factor reduces punishments to tail categories w.r.t. the ratio of cumulative training instances between different categories. Meanwhile, the compensation factor increases the penalty of misclassified instances to avoid false positives of tail categories. We conduct extensive experiments on Seesaw Loss with mainstream frameworks and different data sampling strategies. With a simple end-to-end training pipeline, Seesaw Loss obtains significant gains over Cross-Entropy Loss, and achieves state-of-the-art performance on LVIS dataset without bells and whistles. Code is available at https://github.com/open-mmlab/mmdetection.
Double image encryption algorithm based on neural network and chaos

To realize the secure transmission of double images, this paper proposes a double image encryption algorithm based on convolutional neural network (CNN) and dynamic adaptive diffusion. This scheme is different from the existing double image encryption technology. According to the characteristics of digital image, we design a dual-channel (digital channel / optical channel) encryption method, which not only ensures the security of double image, but also improves the encryption efficiency and reduces the possibility of being attacked. First, a chaotic map is used to control the initial values of the 5D conservative chaotic system to enhance the security of the key. Secondary, in order to effectively resist known-plaintext attack and chosen-plaintext attack, we employ a chaotic sequence as convolution kernel of convolution neural network to generate plaintext related chaotic pointer to control the scrambling operation of two images. On this basis, a novel image fusion method is designed, which splits and fuses two images into two different parts according to the amount of information contained. In addition, a dual-channel image encryption scheme, optical encryption channel and digital encryption channel, is designed for the two parts after fusion. The former has better parallelism and higher encryption efficiency, while the latter has higher computational complexity and better encryption reliability. Especially in the digital encryption channel, a new dynamic adaptive diffusion method is designed, which is more flexible and secure than the existing encryption algorithm. Finally, numerical simulation and experimental analysis verify the feasibility and effectiveness of the scheme.
Neighborhood linear discriminant analysis

Linear Discriminant Analysis (LDA) assumes that all samples from the same class are independently and identically distributed (i.i.d.). LDA may fail in the cases where the assumption does not hold. Particularly when a class contains several clusters (or subclasses), LDA cannot correctly depict the internal structure as the scatter matrices that LDA relies on are defined at the class level. In order to mitigate the problem, this paper proposes a neighborhood linear discriminant analysis (nLDA) in which the scatter matrices are defined on a neighborhood consisting of reverse nearest neighbors. Thus, the new discriminator does not need an i.i.d. assumption. In addition, the neighborhood can be naturally regarded as the smallest subclass, for which it is easier to be obtained than subclass without resorting to any clustering algorithms. The projected directions are sought to make sure that the within-neighborhood scatter as small as possible and the between-neighborhood scatter as large as possible, simultaneously. The experimental results show that nLDA performs significantly better than previous discriminators, such as LDA, LFDA, ccLDA, LM-NNDA, and l2,1-RLDA.
HuMoR: 3D Human Motion Model for Robust Pose Estimation

We introduce HuMoR: a 3D Human Motion Model for Robust Estimation of temporal pose and shape. Though substantial progress has been made in estimating 3D human motion and shape from dynamic observations, recovering plausible pose sequences in the presence of noise and occlusions remains a challenge. For this purpose, we propose an expressive generative model in the form of a conditional variational autoencoder, which learns a distribution of the change in pose at each step of a motion sequence. Furthermore, we introduce a flexible optimization-based approach that leverages HuMoR as a motion prior to robustly estimate plausible pose and shape from ambiguous observations. Through extensive evaluations, we demonstrate that our model generalizes to diverse motions and body shapes after training on a large motion capture dataset, and enables motion reconstruction from multiple input modalities including 3D keypoints and RGB(-D) videos. See the project page at geometry.stanford.edu/projects/humor.
RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching

We introduce RAFT-Stereo, a new deep architecture for rectified stereo based on the optical flow network RAFT [35]. We introduce multi-level convolutional GRUs, which more efficiently propagate information across the image. A modified version of RAFT-Stereo can perform accurate real-time inference. RAFT-stereo ranks first on the Middlebury leaderboard, outperforming the next best method on 1px error by 29% and outperforms all published work on the ETH3D two-view stereo benchmark. Code is available at https://github.com/princeton-vl/RAFT-Stereo.
Quantum gradient descent for linear systems and least squares

Quantum machine learning and optimization are exciting new areas that have been brought forward by the breakthrough quantum algorithm of Harrow, Hassidim, and Lloyd for solving systems of linear equations. The utility of classical linear system solvers extends beyond linear algebra as they can be leveraged to solve optimization problems using iterative methods like gradient descent. In this work, we provide a quantum method for performing gradient descent when the gradient is an affine function. Performing $\ensuremath{\tau}$ steps of the gradient descent requires time $O(\ensuremath{\tau}{C}_{S})$ for weighted least-squares problems, where ${C}_{S}$ is the cost of performing one step of the gradient descent quantumly, which at times can be considerably smaller than the classical cost. We illustrate our method by providing two applications: first, for solving positive semidefinite linear systems, and, second, for performing stochastic gradient descent for the weighted least-squares problem with reduced quantum memory requirements. We also provide a quantum linear system solver in the QRAM data structure model that provides significant savings in cost for large families of matrices.
BoxCars: Improving Fine-Grained Recognition of Vehicles Using 3-D Bounding Boxes in Traffic Surveillance

In this paper, we focus on fine-grained recognition of vehicles mainly in traffic surveillance applications. We propose an approach that is orthogonal to recent advancements in fine-grained recognition (automatic part discovery and bilinear pooling). In addition, in contrast to other methods focused on fine-grained recognition of vehicles, we do not limit ourselves to a frontal/rear viewpoint, but allow the vehicles to be seen from any viewpoint. Our approach is based on 3-D bounding boxes built around the vehicles. The bounding box can be automatically constructed from traffic surveillance data. For scenarios where it is not possible to use precise construction, we propose a method for an estimation of the 3-D bounding box. The 3-D bounding box is used to normalize the image viewpoint by “unpacking” the image into a plane. We also propose to randomly alter the color of the image and add a rectangle with random noise to a random position in the image during the training of convolutional neural networks (CNNs). We have collected a large fine-grained vehicle data set BoxCars116k, with 116k images of vehicles from various viewpoints taken by numerous surveillance cameras. We performed a number of experiments, which show that our proposed method significantly improves CNN classification accuracy (the accuracy is increased by up to 12% points and the error is reduced by up to 50% compared with CNNs without the proposed modifications). We also show that our method outperforms the state-of-the-art methods for fine-grained recognition.
A Novel Real-Time Moving Target Tracking and Path Planning System for a Quadrotor UAV in Unknown Unstructured Outdoor Scenes

A quadrotor unmanned aerial vehicle (UAV) should have the ability to perform real-time target tracking and path planning simultaneously even when the target enters unstructured scenes, such as groves or forests. To accomplish this task, a novel system framework is designed and proposed to accomplish simultaneous moving target tracking and path planning by a quadrotor UAV with an onboard embedded computer, vision sensors, and a two-dimensional laser scanner. A support vector machine-based target screening algorithm is deployed to select the correct target from multiple candidates detected by single shot multibox detector. Furthermore, a new tracker named TLD-KCF is presented in this paper, in which a conditional scale adaptive algorithm is adopted to improve the tracking performance for a quadrotor UAV in cluttered outdoor environments. According to distance and position estimation for a moving target, our quadrotor UAV can acquire a control point to guide its fight. To reduce the computational burden, a fast path planning algorithm is proposed based on elliptical tangent model. A series of experiments are conducted on our quadrotor UAV platform DJI M100. Experimental video and comparison results among four kinds of target tracking algorithms are given to show the validity and practicality of the proposed approach.
Hedging Deep Features for Visual Tracking

Convolutional Neural Networks (CNNs) have been applied to visual tracking with demonstrated success in recent years. Most CNN-based trackers utilize hierarchical features extracted from a certain layer to represent the target. However, features from a certain layer are not always effective for distinguishing the target object from the backgrounds especially in the presence of complicated interfering factors (e.g., heavy occlusion, background clutter, illumination variation, and shape deformation). In this work, we propose a CNN-based tracking algorithm which hedges deep features from different CNN layers to better distinguish target objects and background clutters. Correlation filters are applied to feature maps of each CNN layer to construct a weak tracker, and all weak trackers are hedged into a strong one. For robust visual tracking, we propose a hedge method to adaptively determine weights of weak classifiers by considering both the difference between the historical as well as instantaneous performance, and the difference among all weak trackers over time. In addition, we design a Siamese network to define the loss of each weak tracker for the proposed hedge method. Extensive experiments on large benchmark datasets demonstrate the effectiveness of the proposed algorithm against the state-of-the-art tracking methods.
Toward a Comprehensive Insight Into the Eclipse Attacks of Tor Hidden Services

Tor hidden services (HSs) are used to provide anonymity services to users on the Internet without disclosing the location of the servers so as to enable freedom of speech. However, existing Tor HSs use decentralized architecture that makes it easier for an adversary to launch DHT-based attacks. In this paper, we present practical Eclipse attacks on Tor HSs that allow an adversary with an extremely low cost to block arbitrary Tor HSs. We found that the dominant cost of this attack is IP address resources, the experimental results show that we can use only three IP addresses to eclipse an arbitrary HS with 100% success probability. To understand the severity of the Eclipse attack problems on Tor HSs, and its security implications, we present the first formal analysis to evaluate the extent of threat such vulnerabilities may cause and quantify the costs of Eclipse attacks involved in our attack via probabilistic analysis. Theoretical analysis suggests that adversaries with a modest number of IP address resources can block a large number of HSs at any time. Finally, we discuss countermeasures and future works.
Denoising of 3D magnetic resonance images using a residual encoder–decoder Wasserstein generative adversarial network

Structure-preserved denoising of 3D magnetic resonance imaging (MRI) images is a critical step in medical image analysis. Over the past few years, many algorithms with impressive performances have been proposed. In this paper, inspired by the idea of deep learning, we introduce an MRI denoising method based on the residual encoder-decoder Wasserstein generative adversarial network (RED-WGAN). Specifically, to explore the structure similarity between neighboring slices, a 3D configuration is utilized as the basic processing unit. Residual autoencoders combined with deconvolution operations are introduced into the generator network. Furthermore, to alleviate the oversmoothing shortcoming of the traditional mean squared error (MSE) loss function, the perceptual similarity, which is implemented by calculating the distances in the feature space extracted by a pretrained VGG-19 network, is incorporated with the MSE and adversarial losses to form the new loss function. Extensive experiments are implemented to assess the performance of the proposed method. The experimental results show that the proposed RED-WGAN achieves performance superior to several state-of-the-art methods in both simulated and real clinical data. In particular, our method demonstrates powerful abilities in both noise suppression and structure preservation.
Fast Adaptive Bilateral Filtering

In the classical bilateral filter, a fixed Gaussian range kernel is used along with a spatial kernel for edge-preserving smoothing. We consider a generalization of this filter, the so-called adaptive bilateral filter, where the center and width of the Gaussian range kernel is allowed to change from pixel to pixel. Though this variant was originally proposed for sharpening and noise removal, it can also be used for other applications such as artifact removal and texture filtering. Similar to the bilateral filter, the brute-force implementation of its adaptive counterpart requires intense computations. While several fast algorithms have been proposed in the literature for bilateral filtering, most of them work only with a fixed range kernel. In this paper, we propose a fast algorithm for adaptive bilateral filtering, whose complexity does not scale with the spatial filter width. This is based on the observation that the concerned filtering can be performed purely in range space using an appropriately defined local histogram. We show that by replacing the histogram with a polynomial and the finite range-space sum with an integral, we can approximate the filter using analytic functions. In particular, an efficient algorithm is derived using the following innovations: the polynomial is fitted by matching its moments to those of the target histogram (this is done using fast convolutions), and the analytic functions are recursively computed using integration-by-parts. Our algorithm can accelerate the brute-force implementation by at least $20 \times$, without perceptible distortions in the visual quality. We demonstrate the effectiveness of our algorithm for sharpening, JPEG deblocking, and texture filtering.
Machine learning based privacy-preserving fair data trading in big data market

In the era of big data, the produced and collected data explode due to the emerging technologies and applications that pervade everywhere in our daily lives, including internet of things applications such as smart home, smart city, smart grid, e-commerce applications and social network. Big data market can carry out efficient data trading, which provides a way to share data and further enhances the utility of data. However, to realize effective data trading in big data market, several challenges need to be resolved. The first one is to verify the data availability for a data consumer. The second is privacy of a data provider who is unwilling to reveal his real identity to the data consumer. The third is the payment fairness between a data provider and a data consumer with atomic exchange. In this paper, we address these challenges by proposing a new blockchain-based fair data trading protocol in big data market. The proposed protocol integrates ring signature, double-authentication-preventing signature and similarity learning to guarantee the availability of trading data, privacy of data providers and fairness between data providers and data consumers. We show the proposed protocol achieves the desirable security properties that a secure data trading protocol should have. The implementation results with Solidity smart contract demonstrate the validity of the proposed blockchain-based fair data trading protocol.
An Energy-Efficient Transaction Model for the Blockchain-Enabled Internet of Vehicles (IoV)

The blockchain is a safe, reliable and innovative mechanism for managing numerous vehicles seeking connectivity. However, following the principles of the blockchain, the number of transactions required to update ledgers pose serious issues for vehicles as these may consume the maximum available energy. To resolve this, an efficient model is presented in this letter which is capable of handling the energy demands of the blockchain-enabled Internet of Vehicles (IoV) by optimally controlling the number of transactions through distributed clustering. Numerical results suggest that the proposed approach is 40.16% better in terms of energy conservation and 82.06% better in terms of the number of transactions required to share the entire blockchain-data compared with the traditional blockchain.
A survey on image tampering and its detection in real-world photos

Editing a real-world photo through computer software or mobile applications is one of the easiest things one can do today before sharing the doctored image on one’s social networking sites. Although most people do it for fun, it is suspectable if one concealed an object or changed someone’s face within the image. Before questioning the intention behind the editing operations, we need to first identify how and which part of the image has been manipulated. It therefore demands automatic tools for identifying the intrinsic difference between authentic images and tampered images. This survey provides an overview on typical image tampering types, released image tampering datasets and recent tampering detection approaches. It presents a distinct perspective to rethink various assumptions of tampering clues behind different detection approaches. And this further encourages the research community to develop general tampering localization methods in the future instead of adhering to single-type tampering detection.
Portfolio management with cryptocurrencies: The role of estimation risk

This paper contributes to the literature on cryptocurrencies, portfolio management and estimation risk by comparing the performance of naïve diversification, Markowitz diversification and the advanced Black–Litterman model with VBCs that controls for estimation errors in a portfolio of cryptocurrencies. We show that the advanced Black–Litterman model with VBCs yields superior out-of-sample risk-adjusted returns as well as lower risks. Our results are robust to the inclusion of transaction costs and short-selling, indicating that sophisticated portfolio techniques that control for estimation errors are preferred when managing cryptocurrency portfolios.
A novel directional and non-local-convergent particle swarm optimization based workflow scheduling in cloud–edge environment

With the increasing popularity of Internet of Things (IoT), edge computing has become the key driving force to provide computing resources, storage and network services closer to the edge on the basis of cloud computing. Workflow scheduling in such distributed environment is regarded as an NP-hard problem, and the existing approaches may not work well for task scheduling with multiple optimization goals in complex applications. As an intelligent algorithm, particle swarm optimization (PSO) has the advantages of fewer parameters, simpler algorithm and faster convergence speed, which is widely applied to workflow scheduling. However, there are also some shortcomings such as easy to fall into local optimum and sometimes difficult to obtain real optimal solution. To address this issue, first, the scheduling problem of workflow applications and objective function based on two optimized factors are clearly formalized, which can provide a theoretical foundation for workflow scheduling strategy. Then this paper proposes a novel directional and non-local-convergent particle swarm optimization (DNCPSO) that employs non-linear inertia weight with selection and mutation operations by directional search process, which can reduce the makespan and cost dramatically and obtain a compromising result. The results of simulation experiments based on various real and random workflow examples show that our DNCPSO can achieve better performance than other classical and improved algorithms, which sufficiently demonstrate the effectiveness and efficiency of DNCPSO.
Active Learning From Imbalanced Data: A Solution of Online Weighted Extreme Learning Machine

It is well known that active learning can simultaneously improve the quality of the classification model and decrease the complexity of training instances. However, several previous studies have indicated that the performance of active learning is easily disrupted by an imbalanced data distribution. Some existing imbalanced active learning approaches also suffer from either low performance or high time consumption. To address these problems, this paper describes an efficient solution based on the extreme learning machine (ELM) classification model, called active online-weighted ELM (AOW-ELM). The main contributions of this paper include: 1) the reasons why active learning can be disrupted by an imbalanced instance distribution and its influencing factors are discussed in detail; 2) the hierarchical clustering technique is adopted to select initially labeled instances in order to avoid the missed cluster effect and cold start phenomenon as much as possible; 3) the weighted ELM (WELM) is selected as the base classifier to guarantee the impartiality of instance selection in the procedure of active learning, and an efficient online updated mode of WELM is deduced in theory; and 4) an early stopping criterion that is similar to but more flexible than the margin exhaustion criterion is presented. The experimental results on 32 binary-class data sets with different imbalance ratios demonstrate that the proposed AOW-ELM algorithm is more effective and efficient than several state-of-the-art active learning algorithms that are specifically designed for the class imbalance scenario.