Reviewerly/exHarmony huggingface dataset:
A Learnable Scatternet: Locally Invariant Convolutional Layers

In this paper, we explore tying together the ideas from Scattering Transforms and Convolutional Neural Networks (CNN) for Image Analysis by proposing a learnable ScatterNet. Previous attempts at tying them together in hybrid networks have tended to keep the two parts separate, with the ScatterNet forming a fixed front end and a CNN forming a learned backend. We instead look at adding learning between scattering orders, as well as adding learned layers before the ScatterNet. We do this by breaking down the scattering orders into single convolutional-like layers we call `locally invariant' layers, and adding a learned mixing term to this layer. Our experiments show that these locally invariant layers can improve accuracy when added to either a CNN or a ScatterNet. We also discover some surprising results in that the ScatterNet may be best positioned after one or more layers of learning rather than at the front of a neural network.
Text Detection for Natural Scene based on MobileNet V2 and U-Net

At present, the accuracy of text detection from natural scene images based on deep learning is much higher than that of traditional text detection algorithm. However, most of the text detectors based on deep learning should use large-scale neural network models (such as VGG and ResNet) for improving the accuracy. Because the weight files of these models are very large (VGG is about 500MB and ResNet is about 100MB), they are not suitable for the computationally limited platforms in daily life. Based on the small-scale neural network model named MobileNet V2, and U-Net that is used in semantic segmentation commonly, a text detector for natural scene by using neural network with low complexity is proposed in this paper. The weight file of the designed neural network model is about 16MB, which is suitable for running on mobile platforms. The experiment result based on the ICDAR 2013 dataset proves that this model has good performance in text detection.
A sparse structure for fast circle detection

In the paper, we present a circle detector that achieves the state-of-art performance in almost every type of image. The detector represents each circle instance by a set of equally distributed arcs and searches for the same number of edge points to cover these arcs. The new formulation leads to the voting in minimizing/maximizing way which is different from the typical accumulative way adopted by Hough transform. From the formulation, circle detection is then decomposed into radius-dependent and -independent part. The calculation of independent part is computationally expensive but shared by different radii. This decomposition gets rid of the redundant computation in handling multiple radii and therefore speeds up the detection process. Originated from the sparse nature of independent part, we design a sparse structure for its batch computation which is fulfilled in just one sweep of the edge points. Circle detector based on this sparse structure is then proposed which achieves the comparable time complexity as the algorithm based on Hough transform using 2D accumulator array. For testing, we created an information-rich dataset with images coming from multiple sources. It contains five categories and covers a wide spectrum of images, ranging from true color images to the binary ones. The experimental results demonstrate that the proposed approach outperforms the solutions based on accumulative voting.
Single Image Colorization Via Modified Cyclegan

In this paper, we focus on automatically colorizing single grayscale image without manual interventions. Most of existing methods tried to accurately restore unknown ground-truth colors and require paired training data for model optimization. However, the ideal restoration objective and strict training constraints limited their performance. Inspired by CycleGAN, we formulate the process of colorization as image-to-image translation and propose an effective color-CycleGAN solution. High-level semantic identity loss and low-level color loss are additionally suggested for model optimization. Our method allows using unpaired images for training and direct prediction in rgb color space, which makes training data collection much easier and more general. We train our model on randomly selected PASCAL VOC 2007 images. All ablation study on loss function and comparisons with state-of-the-art methods are performed on grayscale SUN data. The experiment results show that our improvements on training loss could achieve better content consistence and generate better reasonable colors with less artifacts. Moreover, due to the bidirectional nature of our model, our proposed method provides a by-product that gives an excellent alternative way on color image graying.
Learning Depth Cues from Focal Stack for Light Field Depth Estimation

Deep neural networks have shown their excellent abilities in light field depth estimation. Most of learning based approaches focus on the depth feature extraction from the epipolar plane images (EPIs) or sub-apertures of light field, while pay less attention to the focal stack which is also one of the most distinctive characteristics of light field. In this paper, we propose a FocalStackNet which learns depth semantic features and local structure information from the focal stack for light field depth estimation. Specifically, we formulate the disparity estimation as a pixel-wise classification task, and discretize the continuous disparity range into 115 bins. Then we generate a discrete focal stack and extract a set of focal stack patches as training data. Finally, we train a two-pathway convolutional neural networks (CNN) to predict the disparity label of each pixel. Evaluation experiments are carried on the public 4D light field synthetic dataset. Our method achieves state-of-the-art performance. It ranks first among the published methods on the aspects of average and median error scores of Bad Pixel Ratio 0.03.
Pedestrian Group Detection in Shared Space

In shared space, pedestrians are often found walking in groups and behaving differently than individual pedestrians. However, automatically detecting pedestrian groups with high accuracy is not trivial given the dynamic environment and interactions in mixed traffic. Instead of tedious manual work and in order to cope with large scales of data, we propose a time-sequence Density-Based Spatial Clustering of Applications with Noise (DBSCAN) for pedestrian group detection. It is based on coexisting time and Euclidean distance between pedestrians. Our approach outputs reliable results with high IoU values. It can be easily adapted to other groups, e.g., cyclists and animals. In addition to individual behavior, the output data with differentiation of group behavior can be used in further studies in intent detection and motion prediction.
High-Resolution Class Activation Mapping

Insufficient reasoning for their predictions has for long been a major drawback of neural networks and has proved to be a major obstacle for their adoption by several fields of application. This paper presents a framework for discriminative localization, which helps shed some light into the decision-making of Convolutional Neural Networks (CNN). Our framework generates robust, refined and high-quality Class Activation Maps, without impacting the CNN's performance.
Layered Conceptual Image Compression Via Deep Semantic Synthesis

Motivated by the insight of Marr on generative image representations, we propose a layered conceptual image compression scheme by integrating the advantages of both variational auto-encoders (VAEs) and generative adversarial networks (GANs). In particular, the image is represented by two layers: the low-dimensional codes of the stochastic textures encoded by the VAE and the geometric structures characterized by edge maps. Subsequently, the edge maps and latent codes are compressed individually such that the final bit streams are formed in a combined manner. At the decoder side, the GAN synthesizes the decoded images on the basis of the latent codes and the reconstructed edge maps. Experimental results demonstrate that our proposed scheme achieves better visual reconstruction quality than the traditional image compression algorithms such as JPEG, JPEG2000 and HEVC (intra coding) in the low bit rate coding scenarios.
Using Healthcare Authority and Patient Blockchains to Develop a Tamper-Proof Record Tracking System

Blockchain technology combines the openness of the Internet with the security of cryptography to provide a way to verify key information and to establish a trust in applications such as peer-to-peer payment services. Blockchain can be used to track the healthcare transactions and can ensure authorized access to patient records and compliance with the health information protection laws. In this paper, a dual blockchain model for the healthcare sector is proposed. The model combines healthcare authority blockchains and private patient blockchains for a tamperproof permission tracking system which ensures increased security and privacy while improving record and permission redundancy. The proposed model and its possible implementation are discussed in the context of Canadian healthcare privacy legislation in Alberta.
An Improved Hand Gesture Recognition with Two-Stage Convolution Neural Networks Using a Hand Color Image and its Pseudo-Depth Image

Robust hand gesture recognition has been playing a significant role in the field of human-computer interaction for a long time, but it is still full of challenges due to many accept such as cluttered backgrounds and hand self-occlusion. With the help of depth information, depth-based methods have better performance, but the depth cameras are not as widely used and affordable as color cameras. Therefore, in this paper, we propose a two-stage deep convolutional neural network (CNN) architecture for accurate color-based hand gesture recognition. The first stage performs generation of pseudo-depth hand images from color images and the second stage recognizes hand gesture classes using both the color image and its pseudo-depth hand image. The generation stage architecture is based on an image-to-image translation network. In the recognition stage, a two-stream CNN architecture with color image and its pseudo depth image is proposed to improve the color image-based recognition performance. We also propose two strategies in two-stream fusion: feature fusion and committee fusion. To validate our approach, we construct a new dataset called MaHG-RGBD dataset. Experiments demonstrate that our approach significantly improves the performance in RGB-only recognition for hand gestures.
Continuous Sign Language Recognition via Reinforcement Learning

In this paper, we propose an approach to apply the Transformer with reinforcement learning (RL) for continuous sign language recognition (CSLR) task. The Transformer has an encoder-decoder structure, where the encoder network encodes the sign video into the context vector representation, while the decoder network generates the target sentence word by word based on the context vector. To avoid the intrinsic defects of supervised learning (SL) in our task, e.g., the exposure bias and non-differentiable task metrics issues, we propose to train the Transformer directly on non-differentiable metrics, i.e., word error rate (WER), through RL. Moreover, a policy gradient algorithm with baseline, which we call Self-critic REINFORCE, is employed to reduce variance while training. Experimental results on RWTH-PHOENIX-Weather benchmark verify the effectiveness of our method and demonstrate that our method achieves the comparable performance.
Fully Conventional Anchor-Free Siamese Networks for Object Tracking

Siamese network have been extensively applied in the tracking field because of its huge speed advantage and great precision performance in solving the tracking problems.In this paper, we propose an efficient framework for real-time object tracking which is end-to-end trained offline-Fully Conventional Anchor-Free Siamese network (FCAF).Specifically, as the backbone network in Siamese trackers is relatively shallow, resulting in insufficient feature information acquired by the trackers and lower accuracy, the deep network ResNet-50 is adopted to provide richer feature representation.Meanwhile, the introduction of multi-layer feature fusion module effectively combines low-level detail information with high-level semantic features, improving the localization performance.In addition, we propose the anchor-free proposal network (AFPN) to replace the region proposal network (RPN).AFPN network consists of correlation section, implemented by depth-wise cross correlation, and supervised section which has two branches, one for classification and the other for regression.In order to suppress the prediction of low quality bounding boxes, center-ness branch is added.We conduct extensive experiments on the OTB2015 and VOT2016 public datasets, demonstrating that our proposed tracker achieves state-of-the-art performance.
Beyond Synthetic Data: A Blind Deraining Quality Assessment Metric Towards Authentic Rain Image

Deraining quality assessment (DQA) plays an important role in evaluating and guiding the design of the image deraining algorithm. Due to the absence of rain-free image in the real rainy weather, the existing deraining algorithms are typically tested on several synthetic data by simulating very limited types of rain streaks, which are far from sufficient to measure the practicability of a deraining algorithm. In this paper, we first build a subjective DQA database that collects diverse authentic rain images and their derained versions. Then, a blind quality metric is developed to predict the deraining quality. Since the deraining artifacts are anisotropic and variable, we propose to describe the image via a bi-directional gated fusion network (B-GFN), which adaptively integrates the multi-scale cues of deraining artifact. Experiments confirm the effectiveness of the proposed method and its superiority with respect to many state-of-the-art blind image quality metrics.
A component-based video content representation for action recognition

This paper investigates the challenging problem of action recognition in videos and proposes a new component-based approach for video content representation. Although satisfactory performance for action recognition has already been obtained for certain scenarios, many of the existing solutions require fully-annotated video datasets in which region of the activity in each frame is specified by a bounding box. Another group of methods require auxiliary techniques to extract human-related areas in the video frames before being able to accurately recognize actions. In this paper, a Weakly-Supervised Learning (WSL) framework is introduced that eliminates the need for per-frame annotations and learns video representations that improve recognition accuracy and also highlights the activity related regions within each frame. To this end, two new representation ideas are proposed, one focus on representing the main components of an action, i.e. actionness regions, and the other focus on encoding the background context to represent general and holistic cues. A three-stream CNN is developed, which takes the two proposed representations and combines them with a motion-encoding stream. Temporal cues in each of the three different streams are modeled through LSTM, and finally fully-connected neural network layers are used to fuse various streams and produce the final video representation. Experimental results on four challenging datasets, demonstrate that the proposed Component-based Multi-stream CNN model (CM-CNN), trained on a WSL setting, outperforms the state-of-the-art in action recognition, even the fully-supervised approaches.
A new chain coding mechanism for compression stimulated by a virtual environment of a predator–prey ecosystem

In this paper, the researcher introduces a new chain coding mechanism that employs a predator–prey agent-based modeling simulation with some modifications and simplifications and uses it in compression. In the proposed method, an image is ultimately represented by a virtual world consisting of three agent types: wolves, sheep, and paths. While sheep and paths do not change their locations during the program execution, wolves search for sheep to prey on. With each step, a wolf can determine its path by choosing between seven pertinent moves depending on the encountered information. The algorithm keeps track of the wolves’ initial locations and their movements and uses this as a new image representation. Additionally, the researcher introduces the ‘Lengthy Advance Move,’ the purpose of which is to group particular consecutive codes and further reduce the chain. This, in turn, allows the researcher to experiment with different variations of the algorithm. Finally, the researcher applies arithmetic coding on the series of movements for extra compression. The experimental results reveal that the current algorithm generates higher compression ratios than many standardized algorithms, including JBIG family algorithms. Most importantly, paired-sample t-tests reveal significant differences between the findings of the wolf-sheep predation algorithm and the other algorithms used as benchmarks for comparisons.
Accuracy Improvement of Semantic Segmentation Using Appropriate Datasets for Robot Navigation

The use of detailed metric maps for autonomous movement of robots has been popularized in recent times. Three-dimensional sensing devices, such as 3D LiDAR and RADAR, which are expensive yet indispensable, are utilized to generate these metric maps, and ultimately, perform localization. To reduce the cost of sensing devices, we try to realize autonomous movement of a robot using only cheap image sensors, such as webcams. For robot navigation, image processing tends to be applied to collision avoidance by finding obstacles. In contrast, our approach does not use visual object detection but adopts path planning based on movable area extraction from input images using semantic segmentation. To obtain accurate results for visual navigation, this paper proposes the use of novel datasets for semantic segmentation. Experimental results showed that ICNet could extract the movable area with more than 99% accuracy if it was trained with appropriate datasets, and a robot can run automatically based on the extracted movable area.
Towards Multi-source Adaptive Semantic Segmentation

When applying powerful deep learning approaches on real world tasks like pixel level annotation of urban scenes it becomes clear that even those strong learners may fail dramatically and are still not ready for deployment in the wild. For semantic segmentation, one of the main practical challenges consists in finding large annotated collection to feed the data hungry networks. Synthetic images in combination with adaptive learning models have shown to help with this issue, but in general, different synthetic sources are analyzed separately, not leveraging on the potential growth in data amount and sample variability that could result from their combination. With our work we investigate for the first time the multi-source adaptive semantic segmentation setting, proposing some best practice rule for the data and model integration. Moreover we show how to extend an existing semantic segmentation approach to deal with multiple sources obtaining promising results.
A framework for suspect face retrieval using linguistic descriptions

Facial sketch plays a vital role in suspect's identification and apprehension by law enforcement agencies. The sketch artist develops sketches based on the memory of the eye witness. However, sketch artists are less in number and limited to availability. It is also observed that as time passes, the onlooker even forgets many of the critical attributes, which proves costly in time-sensitive investigations. State-of-the-art sketch-photo retrieval methods have used the sketch to retrieve the suspect's photograph and ignore the importance of time sensitivity. In this paper, we have proposed a linguistic description based approach for suspect's photo retrieval. In the proposed approach, the facial attributes and their descriptions are extracted from the input linguistic information using parts of speech and attributes ontology. Attributes are ranked according to their importance in the identification of a suspect. An ensemble classifier which aggregates the predictions of multiple classifiers is used to retrieve a suspect's image. Experiments are performed on standard datasets. The performance of the proposed approach is evaluated by comparing it with the existing methods of linguistic sketch-based retrieval as well as with sketch to photo retrieval method. Experimental results illustrate that the proposed linguistic-base method using ensemble classification attains auspicious performance as compared to state-of-the-art methods.
Exploring MPI Communication Models for Graph Applications Using Graph Matching as a Case Study

Traditional implementations of parallel graph operations on distributed memory platforms are written using Message Passing Interface (MPI) point-to-point communication primitives such as Send-Recv (blocking and nonblocking). Apart from this classical model, the MPI community has over the years added other communication models; however, their suitability for handling the irregular traffic workloads typical of graph operations remain comparatively less explored. Our aim in this paper is to study these relatively underutilized communication models of MPI for graph applications. More specifically, we evaluate MPI's one-sided programming, or Remote Memory Access (RMA), and nearest neighborhood collectives using a process graph topology. There are features in these newer models that are intended to better map to irregular communication patterns, as exemplified in graph algorithms. As a concrete application for our case study, we use distributed memory implementations of an approximate weighted graph matching algorithm to investigate performances of MPI3 RMA and neighborhood collective operations compared to nonblocking Send-Recv. A matching in a graph is a subset of edges such that no two matched edges are incident on the same vertex. A maximum weight matching is a matching of maximum weight computed as the sum of the weights of matched edges. Execution of graph matching is dominated by high volume of irregular memory accesses, making it an ideal candidate for studying the effects of various MPI communication models on graph applications at scale. Our neighborhood collectives and RMA implementations yield up to 6× speedup over traditional nonblocking Send-Recv implementations on thousands of cores of the NERSC Cori supercomputer. We believe the lessons learned from this study can be adopted to benefit a wider range of graph applications.
A Programmable Architecture for Robot Motion Planning Acceleration

We have designed a programmable architecture to accelerate collision detection and graph search, two of the principal components of robotic motion planning. The programmability enables the architecture to be applied to a wide range of different robots and motion planning applications. We present the architecture of our accelerator and describe and evaluate its microarchitecture implementation.