Reviewerly/exHarmony huggingface dataset:
A divide-and-conquer strategy for facial landmark detection using dual-task CNN architecture

In this paper, we propose a novel deep learning-based framework for facial landmark detection. This framework takes as input face image returned by a face detector (Faster R-CNN) and generates as output a set of landmarks positions. Prior CNN-based methods often select randomly small local patches to predict an initial guess of landmarks locations. One issue with these local patches is that the adjacent landmarks might share the same regions due to the overlapping, thus, they might not convey precise information of each individual landmark. By contrast, our approach formulates this problem as a divide-conquer search for facial patches using CNN architecture in a hierarchy, where the input face image is recursively split into two cohesive non-overlapped subparts until each one contains only the region around the expected landmark. To attain better division of face topology, the search is carried out in a structured coarse-to-fine manner, where a learned hierarchical model of the face defining the granularity of each division level is introduced. We also propose a cascaded regressor to detect and refine the position of the individual landmark in each predicted non-overlapped patch. We adopt a carefully designed shallow CNN architecture so that to improve real-time performance. In addition, unlike previous cascaded methods, our regressor does not require auxiliary input such as initial landmarks locations. Extensive experiments on several challenging datasets (including MTFL, AFW, AFLW, COFW, 300W, and 300VW) show that our approach is particularly impressive in the unconstrained scenarios where it outperforms prior arts in both accuracy and efficiency.
Attention-Based Context Aware Reasoning for Situation Recognition

Situation Recognition (SR) is a fine-grained action recognition task where the model is expected to not only predict the salient action of the image, but also predict values of all associated semantic roles of the action. Predicting semantic roles is very challenging: a vast variety of possibilities can be the match for a semantic role. Existing work has focused on dependency modelling architectures to solve this issue. Inspired by the success achieved by query-based visual reasoning (e.g., Visual Question Answering), we propose to address semantic role prediction as a query-based visual reasoning problem. However, existing query-based reasoning methods have not considered handling of inter-dependent queries which is a unique requirement of semantic role prediction in SR. Therefore, to the best of our knowledge, we propose the first set of methods to address inter-dependent queries in query-based visual reasoning. Extensive experiments demonstrate the effectiveness of our proposed method which achieves outstanding performance on Situation Recognition task. Furthermore, leveraging query inter-dependency, our methods improve upon a state-of-the-art method that answers queries separately. Our code: https://github.com/thilinicooray/context-aware-reasoning-for-sr.
A Blockchain-Based P2P Transaction Method and Sensitive Data Encoding for E-Commerce Transactions

With the rapid development and application of information technology, blockchain technology (BT) has become an important means to promote the integration of the real economy and the digital economy. BT reached a certain level of development for application in asset transactions, finance, and traceability. However, existing technology provides no reasonable trading method and application framework for the use of BT in peer-to-peer (P2P) transactions. This article proposed a blockchain-based P2P transaction method, which we simulated using Go programming language. We apply BT to smart contracts and electronic transactions, and introduce a mechanism of association between digital assets and physical assets. The proposed method can ensure the data security and establish trust among entities. It can also reduce business divergence and costs. For the sensitive data involved in a transaction, we propose an encoding algorithm to prevent the leaks of sensitive data in circulation. The calculations of the matrices of the algorithm can use the computing resources of all nodes in the blockchain to calculate in parallel, and then summarize the results. This makes full use of the massive graphic processing unit resources in the blockchain, and makes the calculation valuable.
The Ethical Balance of Using Smart Information Systems for Promoting the United Nations’ Sustainable Development Goals

The Sustainable Development Goals (SDGs) are internationally agreed goals that allow us to determine what humanity, as represented by 193 member states, finds acceptable and desirable. The paper explores how technology can be used to address the SDGs and in particular Smart Information Systems (SIS). SIS, the technologies that build on big data analytics, typically facilitated by AI techniques such as machine learning, are expected to grow in importance and impact. Some of these impacts are likely to be beneficial, notably the growth in efficiency and profits, which will contribute to societal wellbeing. At the same time, there are significant ethical concerns about the consequences of algorithmic biases, job loss, power asymmetries and surveillance, as a result of SIS use. SIS have the potential to exacerbate inequality and further entrench the market dominance of big tech companies, if left uncontrolled. Measuring the impact of SIS on SDGs thus provides a way of assessing whether an SIS or an application of such a technology is acceptable in terms of balancing foreseeable benefits and harms. One possible approach is to use the SDGs as guidelines to determine the ethical nature of SIS implementation. While the idea of using SDGs as a yardstick to measure the acceptability of emerging technologies is conceptually strong, there should be empirical evidence to support such approaches. The paper describes the findings of a set of 6 case studies of SIS across a broad range of application areas, such as smart cities, agriculture, finance, insurance and logistics, explicitly focusing on ethical issues that SIS commonly raise and empirical insights from organisations using these technologies.
Coarse to Fine Rate Control For Region-Based 3D Point Cloud Compression

We modify the video-based point cloud compression standard (V-PCC) by mapping the patches to seven regions and encoding the geometry and color video sequences of each region. We then propose a coarse to fine rate control algorithm for this scheme. The algorithm consists of two major steps. First, we allocate the target bitrate between the geometry and color information. Then, we optimize in turn the geometry and color quantization steps for the video sequences of each region using analytical models for the rate and distortion. Experimental results for eight point clouds showed that the average percent bitrate error of our algorithm is only 3.7%, and its perceptual reconstruction quality is better than that of V-PCC.
Model-Driven Deep Learning for Massive MU-MIMO With Finite-Alphabet Precoding

Massive multiuser multiple-input multiple-output (MU-MIMO) has been the mainstream technology in fifth-generation wireless systems. To reduce high hardware costs and power consumption in massive MU-MIMO, low-resolution digital-to-analog converters (DAC) for each antenna and radio frequency (RF) chain in downlink transmission is used, which brings challenges for precoding design. To circumvent these obstacles, we develop a model-driven deep learning (DL) network for massive MU-MIMO with finite-alphabet precoding in this article. The architecture of the network is specially designed by unfolding an iterative algorithm. Compared with the traditional state-of-the-art techniques, the proposed DL-based precoder shows significant advantages in performance, complexity, and robustness to channel estimation error under Rayleigh fading channel.
Sequential Motif Profiles and Topological Plots for Offline Signature Verification

In spite of the overwhelming high-tech marvels and applications that rule our digital lives, the use of the handwritten signature is still recognized worldwide in government, personal and legal entities to be the most important behavioral biometric trait. A number of notable research approaches provide advanced results up to a certain point which allow us to assert with confidence that the performance attained by signature verification (SV) systems is comparable to those provided by any other biometric modality. Up to now, the mainstream trend for offline SV is shared between standard -or handcrafted- feature extraction methods and popular machine learning techniques, with typical examples ranging from sparse representation to Deep Learning. Recent progress in graph mining algorithms provide us with the prospect to re-evaluate the opportunity of utilizing graph representations by exploring corresponding graph features for offline SV. In this paper, inspired by the recent use of image visibility graphs for mapping images into networks, we introduce for the first time in offline SV literature their use as a parameter free, agnostic representation for exploring global as well as local information. Global properties of the sparsely located content of the shape of the signature image are encoded with topological information of the whole graph. In addition, local pixel patches are encoded by sequential visibility motifs-subgraphs of size four, to a low six dimensional motif profile vector. A number of pooling functions operate on the motif codes in a spatial pyramid context in order to create the final feature vector. The effectiveness of the proposed method is evaluated with the use of two popular datasets. The local visibility graph features are considered to be highly informative for SV; this is sustained by the corresponding results which are at least comparable with other classic state-of-the-art approaches.
Recursive Least-Squares Estimator-Aided Online Learning for Visual Tracking

Online learning is crucial to robust visual object tracking as it can provide high discrimination power in the presence of background distractors. However, there are two contradictory factors affecting its successful deployment on the real visual tracking platform: the discrimination issue due to the challenges in vanilla gradient descent, which does not guarantee good convergence; the robustness issue due to over-fitting resulting from excessive update with limited memory size (the oldest samples are discarded). Despite many dedicated techniques proposed to somehow treat those issues, in this paper we take a new way to strike a compromise between them based on the recursive least-squares estimation (LSE) algorithm. After connecting each fully-connected layer with LSE separately via normal equations, we further propose an improved mini-batch stochastic gradient descent algorithm for fully-connected network learning with memory retention in a recursive fashion. This characteristic can spontaneously reduce the risk of over-fitting resulting from catastrophic forgetting in excessive online learning. Meanwhile, it can effectively improve convergence though the cost function is computed over all the training samples that the algorithm has ever seen. We realize this recursive LSE-aided online learning technique in the state-of-the-art RT-MDNet tracker, and the consistent improvements on four challenging benchmarks prove its efficiency without additional offline training and too much tedious work on parameter adjusting.
Mcships: A Large-Scale Ship Dataset For Detection And Fine-Grained Categorization In The Wild

This paper introduces a multi-category ship dataset (called McShips), which is a challenging and large-scale dataset aimed at ship detection and fine-grained categorization. The McShips dataset includes 14,709 annotated images of ships belonging to 6 classes of warships and 7 classes of civilian ships. Each image is carefully annotated with a bounding box and ship class label. The dataset is challenging due to the following two reasons: First, there is little inter-class variation as ships have very similar ship shapes; Second, there is very large intra-class variation since the ships within the same class may be significantly different due to viewpoint variations, weather condition variations, illumination variations, scale changes, occlusion, cluttered background and so on. Based on the McShips dataset, we present the detection and fine-grained categorization performance of three baseline detectors on our dataset, and make a comparison to identify the strengths and weaknesses of the baseline detection algorithms. We hope the presented McShips dataset would advance research and applications on ship detection and finegrained categorization.
Single Image Rain Removal Boosting Via Directional Gradient

Image rain removal has been widely studied with traditional methods and learning based methods for years. However, traditional methods like Gaussian mixture model and dictionary learning methods are time consuming and fail to well tackle images with heavy rain streaks since image patches are severely contaminated. By considering the line-like property and angle distribution of rain streaks, this problem can be well solved. In this paper, by introducing Directional b radient operator of arbitrary direction, we propose an efficient and robust Constraints based Model (DiG-CoM) for single image rain removal. Moreover, a density metric of rain streaks is applied to generalize the proposed model to light and heavy rain streak occasions. Extensive experiments on synthetic datasets demonstrate that the proposed model outperforms GMM and JCAS while requiring less time. Furthermore, on real-world occasions, the proposed method obtains better generalization ability compared with the stateof-the-art learning based methods. The source code is available at https://github.com/Schizophreni/Set-vanish-to-the-rain.
A neurodynamic optimization approach for complex-variables programming problem

A neural network model upon differential inclusion is designed for solving the complex-variables convex programming, and the chain rule for real-valued function with the complex-variables is established in this paper. The model does not need to choose penalty parameters when applied to practical problems, which makes it easier to design. The result is obtained that its state reaches the feasible region in finite time. Furthermore, the convergence for its state to an optimal solution is proved. Some typical examples are shown for the effectiveness of the designed model.
Embodied Language Grounding With 3D Visual Feature Representations

We propose associating language utterances to 3D visual abstractions of the scene they describe. The 3D visual abstractions are encoded as 3-dimensional visual feature maps. We infer these 3D visual scene feature maps from RGB images of the scene via view prediction: when the generated 3D scene feature map is neurally projected from a camera viewpoint, it should match the corresponding RGB image. We present generative models that condition on the dependency tree of an utterance and generate a corresponding visual 3D feature map as well as reason about its plausibility, and detector models that condition on both the dependency tree of an utterance and a related image and localize the object referents in the 3D feature map inferred from the image. Our model outperforms models of language and vision that associate language with 2D CNN activations or 2D images by a large margin in a variety of tasks, such as, classifying plausibility of utterances, detecting referential expressions, and supplying rewards for trajectory optimization of object placement policies from language instructions. We perform numerous ablations and show the improved performance of our detectors is due to its better generalization across camera viewpoints and lack of object interferences in the inferred 3D feature space, and the improved performance of our generators is due to their ability to spatially reason about objects and their configurations in 3D when mapping from language to scenes.
Reflection Scene Separation From a Single Image

For images taken through glass, existing methods focus on the restoration of the background scene by regarding the reflection components as noise. However, the scene reflected by glass surface also contains important information to be recovered, especially for the surveillance or criminal investigations. In this paper, instead of removing reflection components from the mixture image, we aim at recovering reflection scenes from the mixture image. We first propose a strategy to obtain such ground truth and its corresponding input images. Then, we propose a two-stage framework to obtain the visible reflection scene from the mixture image. Specifically, we train the network with a shift-invariant loss which is robust to misalignment between the input and output images. The experimental results show that our proposed method achieves promising results.
Dual-domain Deep Convolutional Neural Networks for Image Demoireing

We develop deep convolutional neural networks (CNNs) for moiré artifacts removal by exploiting the complex properties of moiré patterns in multiple complementary domains, i.e., the pixel and frequency domains. In the pixel domain, we employ multi-scale features to remove the moiré artifacts associated with specific frequency bands using multi-resolution feature maps. In the frequency domain, we design a network that processes discrete cosine transform (DCT) coefficients to remove moiré artifacts. Next, we develop a dynamic filter generation network that learns dynamic blending filters. Finally, the results from the pixel and frequency domains are combined using the blending filters to yield moiré-free images. In addition, we extend the proposed approach to arbitrary-length burst image demoireing. Specifically, we develop a new attention network to effectively extract useful information from each image in the burst and align them with the reference image. We demonstrate the effectiveness of the proposed demoireing algorithm by evaluating on the test set in the NTIRE 2020 Demoireing Challenge: Track 1 (Single image) and Track 2 (Burst).
Financial Development, Real Sector and Economic Growth in Sub-Saharan Africa: The Threshold Effect

(2021). Financial Development, Real Sector and Economic Growth in Sub-Saharan Africa: The Threshold Effect. Journal of African Business: Vol. 22, Entrepreneurship and the Informal Sector: Challenges and Opportunities for African Business Development, pp. 603-626.
Self-driving vehicles: current status of development and technical challenges to overcome

Autonomous vehicles (AVs) are the alternative solution for mobility in the future. The first major benefit of the self-driving cars would be the safety. Human errors cause most of the car incidents. Self-driving cars could improve public transportation services and decrease auto ownership because personal car won't be necessary anymore. Self-driving car can be a solution to reduce the emissions of carbon dioxide (CO2). There are also some challenges to overcome: self-driving cars cannot operate very good in bad weather conditions and the legislative structure needs to be defined in case of any incident. This article presents the current status of this technology and what are the challenges of implementing intelligent systems that allow the handling of a vehicle without human intervention.
App trajectory recognition over encrypted internet traffic based on deep neural network

Despite the increasing popularity of mobile applications and the widespread adoption of encryption techniques, mobile devices are still susceptible to security and privacy risks. In this paper, we propose ActiveTracker, a new type of sniffing attack that can reveal the fine-grained trajectory of userâs mobile app usage from a sniffed encrypted Internet traffic stream. It firstly adopts a sliding window based approach to divide the encrypted traffic stream into a sequence of segments corresponding to different app activities. Then each traffic segment is represented by a normalized temporal-spacial traffic matrix and a traffic spectrum vector. Based on the normalized representation, a deep neural network (DNN) model which consists of an app filter and an activity classifier is developed to extract comprehensive features from the input and uncover the crucial app usage trajectory conducted by the user. By extensive experiments on real-world app usage traffic collected from volunteers and on our synthetic traffic data, we show that the proposed approach achieves up to 79.65% accuracy in recognizing app trajectory over encrypted traffic streams.
Cluttered TextSpotter: An End-to-End Trainable Light-Weight Scene Text Spotter for Cluttered Environment

Scene text spotting aims at simultaneously localizing and recognizing text instances, symbols, and logos in natural scene images. Scene text detection and recognition approaches have received immense attention in computer vision research community. The presence of partial occlusion or truncation artifact due to the cluttered background of scene images creates an obstacle in perceiving the text instances, which makes the process of spotting very complex. In this paper, we propose a light-weight scene text spotter that can address the issue of cluttered environment of scene images. It is an end-to-end trainable deep neural network that uses local part information, global structural features, and context cue information of oriented region proposals for spotting text instances. It helps to localize in scene images with background clutters, where partially occluded text parts, truncation artifacts, and perspective distortions are present. We mitigate the problem of misclassification caused by inter-class interference by exploring inter-class separability and intra-class compactness. We also incorporate multi-language character segmentation and word-level recognition in a light-weight recognition module. We have used six publicly available benchmark datasets in different smart devices to illustrate the efficacy of the network.
A tractable latent variable model for nonlinear dimensionality reduction

Significance Latent variable models (LVMs) are powerful tools for discovering hidden structure in data. Canonical LVMs include factor analysis, which explains the correlation of a large number of observed variables in terms of a smaller number of unobserved ones, and Gaussian mixture models, which reveal clusters of data arising from an underlying multimodal distribution. In this paper, we describe a conceptually simple and equally effective LVM for nonlinear dimensionality reduction (NLDR), where the goal is to discover faithful, neighborhood-preserving embeddings of high-dimensional data. Tools for NLDR can help researchers across all areas of science and engineering to better understand and visualize their data. Our approach elevates NLDR into the family of problems that can be studied by especially tractable LVMs.
Estimation of fill factor for earth-moving machines based on 3D point clouds

Bucket fill factor is of paramount importance in measuring the productivity of earth-moving machines which is the amount of material loaded in the bucket within one scooping. In this paper, a stereo vision-based method is proposed and a novel perception system is developed. The system tested on a wheel loader is able to provide an accurate estimation of the fill factor. Disparity is computed for a pair of stereo images to reconstruct high fidelity 3-D point clouds. Preliminary processing is then used to extract regions of interest. In addition to applying existing state-of-the-art techniques including point cloud registration, stitching, and surface interpolation, our system also incorporates the structural information of wheel loaders, achieving excellent performance of 94.72% accuracy even under the unavailability of visual information due to occlusion, and under different environment conditions. This research paves the way for the arrival of fully autonomous earth-moving machines in the future.