Reviewerly/exHarmony huggingface dataset:
A method for detecting text of arbitrary shapes in natural scenes that improves text spotting

Understanding the meaning of text in images of natural scenes like highway signs or store front emblems is particularly challenging if the text is foreshortened in the image or the letters are artistically distorted. We introduce a pipeline-based text spotting framework that can both detect and recognize text in various fonts, shapes, and orientations in natural scene images with complicated backgrounds. The main contribution of our work is the text detection component, which we call UHT, short for UNet, Heatmap, and Textfill. UHT uses a UNet to compute heatmaps for candidate text regions and a textfill algorithm to produce tight polygonal boundaries around each word in the candidate text. Our method trains the UNet with groundtruth heatmaps that we obtain from text bounding polygons provided by groundtruth annotations. Our text spotting framework, called UHTA, combines UHT with the state-of-the-art text recognition system ASTER. Experiments on four challenging and public scene-text-detection datasets (Total-Text, SCUT-CTW1500, MSRA-TD500, and COCO-Text) show the effectiveness and generalization ability of UHT in detecting not only multilingual (potentially rotated) straight but also curved text in scripts of multiple languages. Our experimental results of UHTA on the TotalText dataset show that UHTA outperforms four state-of-the- art text spotting frameworks by at least 9.1 percent points in the F-measure, which suggests that UHTA may be used as a complete text detection and recognition system in real applications.
IU-Module: Intersection and Union Module for Fine-Grained Visual Classification

A predominant viewpoint in previous works of fine-grained visual classification (FGVC) is to the localize discriminative parts by auxiliary networks and extract the part-based finegrained features for classification. In this paper, we propose a simple yet effective approach by introducing an intersection and union module (IU-Module). The IU-Module aims to capture more discriminative features by 1) dividing features into distinct groups, 2) sharing parts of interests within each group, and 3) adding a differentiation loss to reduce the similarity among those grouped feature channels. Without adding any new learnable parameters, the proposed approach imposes two straightforward operations, namely channel intersection (CI) and channel union (CU) operations, on the convolutional features and achieves competitive results compared with the state-of-the-art methods. Experimental results on three publicly available FGVC datasets show the effectiveness of the IU-Module. Ablation studies and visualizations are also provided to make further demonstrations.
ODRE Workshop: Probabilistic Dynamic Hard Real-Time Scheduling in HPC

Industry 4.0 is changing fundamentally the way data is collected, stored and analyzed in industrial processes. While this change enables novel application such as flexible manufacturing of highly customized products, the real-time control of these processes, however, has not yet realized its full potential. We believe that modern virtualization techniques, specifically application containers, present a unique opportunity to decouple control functionality from associated hardware. Through it, we can fully realize the potential for highly distributed and transferable industrial processes even with real-time constraints arising from time-critical sub-processes. In this paper, we present a specifically developed orchestration tool to manage the challenges and opportunities of shifting industrial control software from dedicated hardware to bare-metal servers or (edge) cloud computing platforms. Using off-the-shelf technology, the proposed tool can manage the execution of containerized applications on shared resources without compromising hard real-time execution determinism. Through first experimental results, we confirm the viability and analyzed the behavior of resource shared systems with strict real-time requirements. We then describe experiments set out to deliver expected results and gather performance, application scope and limits of the presented approach.
End-to-End Method For Labanotation Generation From Continuous Motion Capture Data

Labanotation is a standardized notation system for human motion recording and archiving. Existing methods for automatic Labanotation generation require pre-segmentation of continuous motion and only recognize single movement each time. The poor performance of pre-segmentation will subsequently lead to inaccurate movement recognition and Labanotation generation. In this paper, we proposed an end-to-end trainable method for recognizing continuous motion and generating corresponding Laban symbols without presegmentation. Firstly, we design Lie group feature to represent rotation information of joints and bones underlying in continuous motion capture data. Secondly, we employ Convolutional Recurrent Neural Network (CRNN) to jointly analyze motion sequences in spatial and temporal domain. Furthermore, we adopt Connectionist Temporal Classification (CTC) layer to translate per-frame predictions into Laban symbol sequence, which enables analysis on motion sequences with arbitrary lengths. Experiments on continuous motion capture dataset demonstrate the effectiveness of proposed method and its superiority compared with the state-of-the-art.
Information Distribution Based Defense Against Physical Attacks on Object Detection

Recently, physical attacks launch a new challenge to the security of deep neural networks (DNNs) by generating physical-world adversarial patches to attack DNNs based applications. The information distribution contained in the adversarial patch is different from that in the real image patches. In this paper, we propose a general defense method to effectively prevent such attacks. This method consists of an entropy-based proposal component and a gradient-based filtering component. Each component of our method can be viewed as preprocessing of adversarial images. Processed images are then run through the unmodified detectors, making our method agnostic to both the detectors and the attacks. Moreover, our method is based on traditional image processing rather than DNNs, so it does not require a great quantity of training data. Extensive experiments on different datasets indicate that our method is able to defend against physical attack on object detection effectively, increasing mAP from 31.3% to 53.8% for Pascal VOC 2007 and from 19.0% to 40.3% for Inria, and has better transferability, which can defend against different physical attacks.
RRNet: Repetition-Reduction Network for Energy Efficient Depth Estimation

Lightweight neural networks that employ depthwise convolution have a significant computational advantage over those that use standard convolution because they involve fewer parameters; however, they also require more time, even with graphics processing units (GPUs). We propose a Repetition-Reduction Network (RRNet) in which the number of depthwise channels is large enough to reduce computation time while simultaneously being small enough to reduce GPU latency. RRNet also reduces power consumption and memory usage, not only in the encoder but also in the residual connections to the decoder. We apply RRNet to the problem of resource-constrained depth estimation, where it proves to be significantly more efficient than other methods in terms of energy consumption, memory usage, and computation. It has two key modules: the Repetition-Reduction (RR) block, which is a set of repeated lightweight convolutions that can be used for feature extraction in the encoder, and the Condensed Decoding Connection (CDC), which can replace the skip connection, delivering features to the decoder while significantly reducing the channel depth of the decoder layers. Experimental results on the KITTI dataset show that RRNet consumes 3.84x less energy and 3.06x less memory than conventional schemes, and that it is 2.21x faster on a commercial mobile GPU without increasing the demand on hardware resources relative to the baseline network. Furthermore, RRNet outperforms state-of-the-art lightweight models such as MobileNets, PyDNet, DiCENet, DABNet, and EfficientNet.
Glad: Global And Local Anomaly Detection

Detecting anomaly in images is challenging due to the high dimension nature of image data. While the previous learning-based anomaly detection approaches can detect a particular type of anomaly precisely, they often fail in detecting multiple types of abnormal samples simultaneously.We identify the two specific types of anomalies that can be precisely detected by either compress-based or reconstruction-based anomaly detection approaches, named global anomaly and local anomaly. We then propose Glad, an anomaly detector that can precisely detect both of them at the same time. Glad adopts a joint approach combining the density estimation and auto-encoder. Firstly, it designs a multimodal density estimation model to derive the latent representation probability for identifying the global anomaly. Then, it uses structural similarity to measure the reconstruction loss for characterizing local anomaly. Finally, both anomalies can be diagnosed according to the joint density of latent representation and reconstruction loss. Experimental results on public benchmark datasets demonstrate that Glad outperforms the state-of-the-art methods significantly.
EDNC: Evolving Differentiable Neural Computers

One recent study in the field of deep learning is extending Artificial Neural Networks (ANNs) by coupling them to external memory resources. Neural Turing Machine (NTM) and Differentiable Neural Computer (DNC) are two counterparts in this field. Research activities fall into two categories of either interacting with memory or choosing controller components. The first approach uses a try and error fashion to provide the controller's structure that would be a problem when there is no prior knowledge on the controller's structure for particular tasks. The second approach includes methods for choosing the controller's structure and weight optimization. NeuroEvolution falls in the second category that automatically and without prior knowledge obtains an appropriate network structure. This research presents Evolutionary Differentiable Neural Computer (EDNC), which uses a novel NeuroEvolution algorithm that is introduced in two types of nested object-oriented encoding called Adaptive Layer NeuroEvolution (ALNE) and Matrix-based one called M_ALNE. These NeuroEvolution algorithms use the following sub-algorithms: Self-Adaptivity in the Initial Population (SAIP), Self-Adaptivity in Evolutionary Structures (SAES), Layer Recombination (LR), Layer Mutation (LM), and Self-Adaptivity in Mutation (SAM). The evolution process starts with SAIP, and then it takes short and long steps to explore a variety of neural structures by SAES algorithm that uses both LR and LM algorithms to provide a variety of structures. Finally, the SAM algorithm guides the selected structures to a target structure. In this research, experiments applied EDNC on well-known tasks, including Facebook bAbI, copy, graph (shortest path), and 8-puzzle. The experiments show that the proposed method in both encodings automatically and without prior knowledge, produces an appropriate controller structure in the shortest time. It though shows that both proposed encodings reduce the evolution time by at least 73% in comparison with the baseline methods.
A Lightweight Environment for Learning Experimental IR Research Practices

Tools, computing environments, and datasets form the three critical ingredients for teaching and learning the practical aspects of experimental IR research. Assembling these ingredients can often be challenging, particularly in the context of short courses that cannot afford large startup costs. As an initial attempt to address these issues, we describe materials that we have developed for the "Introduction to IR" session at the ACM SIGIR/SIGKDD Africa Summer School on Machine Learning for Data Mining and Search (AFIRM 2020), which builds on three components: the open-source Lucene search library, cloud-based notebooks, and the MS MARCO dataset. We offer a self-reflective evaluation of our efforts and hope that our lessons shared can benefit future efforts.
A Survey on Speech Feature Extraction and Classification Techniques

Speech recognition is an approach of acknowledging human speech with the aid of the system and to produce string output in written format. A model is positioned from a crew of audio recordings whose corresponding transcripts are created with the resource of taking recordings of speech as audio and their textual content transcriptions, the use of software program to create statistical representations of the sounds that would make up every phrase famous via incorporating Language Processing (NLP) methods. For several decades, researchers are working in the field of speech recognition and communication. This paper describes some of the techniques and approaches that are developed by various researchers in the field of speech recognition.
An Improved Crow Search Algorithm for Data Clustering

Metaheuristic algorithms are often trapped in local optimum solutions when searching for solutions. This problem often occurs in optimization cases involving high dimensions such as data clustering. Imbalance of the exploration and exploitation process is the cause of this condition because search agents are not able to reach the best solution in the search space. In this study, the problem is overcome by modifying the solution update mechanism so that a search agent not only follows another randomly chosen search agent, but also has the opportunity to follow the best search agent. In addition, the balance of exploration and exploitation is also enhanced by the mechanism of updating the awareness probability of each search agent in accordance with their respective abilities in searching for solutions. The improve mechanism makes the proposed algorithm obtain pretty good solutions with smaller computational time compared to Genetic Algorithm and Particle Swarm Optimization. In large datasets, it is proven that the proposed algorithm is able to provide the best solution among the other algorithms.
Multi-source Domain Adaptation for Sentiment Classification with Granger Causal Inference

In this paper, we propose a multi-source domain adaptation method with a Granger-causal objective (MDA-GC) for cross-domain sentiment classification. Specifically, for each source domain, we build an expert model by using a novel sentiment-guided capsule network, which captures the domain invariant knowledge that bridges the knowledge gap between the source and target domains. Then, an attention mechanism is devised to assign importance weights to a mixture of experts, each of which specializes in a different source domain. In addition, we propose a Granger causal objective to make the weights assigned to individual experts correlate strongly with their contributions to the decision at hand. Experimental results on a benchmark dataset demonstrate that the proposed MDA-GC model significantly outperforms the compared methods.
ProAlignNet: Unsupervised Learning for Progressively Aligning Noisy Contours

Contour shape alignment is a fundamental but challenging problem in computer vision, especially when the observations are partial, noisy, and largely misaligned. Recent ConvNet-based architectures that were proposed to align image structures tend to fail with contour representation of shapes, mostly due to the use of proximity-insensitive pixel-wise similarity measures as loss functions in their training processes. This work presents a novel ConvNet, "ProAlignNet," that accounts for large scale misalignments and complex transformations between the contour shapes. It infers the warp parameters in a multi-scale fashion with progressively increasing complex transformations over increasing scales. It learns --without supervision-- to align contours, agnostic to noise and missing parts, by training with a novel loss function which is derived an upperbound of a proximity-sensitive and local shape-dependent similarity metric that uses classical Morphological Chamfer Distance Transform. We evaluate the reliability of these proposals on a simulated MNIST noisy contours dataset via some basic sanity check experiments. Next, we demonstrate the effectiveness of the proposed models in two real-world applications of (i) aligning geo-parcel data to aerial image maps and (ii) refining coarsely annotated segmentation labels. In both applications, the proposed models consistently perform superior to state-of-the-art methods.
Blind S3D image quality prediction using classical and non-classical receptive field models

We develop a novel no-reference image quality assessment model for stereoscopic 3D (S3D) images that is inspired by functional receptive field models of perceptual mechanisms in primary visual cortex (V1). The approach is called the Blind S3D Integrated Quality Evaluator (BSIQE). BSIQE simulates monocular and binocular responses to stereo views using channel separation and weighted multi-channel combination models. Binocular responses are modeled as the fusion of the two channels using a weighted multi-channel combination. The responses to stereoscopic image content of both classical and non-classical anisotropic receptive fields are then modeled based on a determination of the relative importance of the receptive field responses. In the last stage of feature extraction, we deploy a simple and efficient way of decorrelating the picture data. We extract local binary pattern (LBP) statistical features from the computed receptive field responses, and use them to train a regressor to predict the perceptual quality of stereoscopic images. We carefully evaluate BSIQE on four public-domain 3D image quality databases, and find that it is statistically superior to all compared 2D and 3D IQA algorithms. BSIQE exhibits good performance across the datasets suggesting that it is general, and it has relatively low complexity.
On Privacy Preserving Anonymization of Finger-selfies

With the availability of smartphone cameras, high speed internet, and connectivity to social media, users post content on the go including check-ins, text, and images. Privacy leaks due to posts related to check-ins and text is an issue in itself, however, this paper discusses the potential leak of one's biometric information via images posted on social media. While posting photos of themselves or highlighting miniature objects, users end up posting content that leads to an irreversible loss of biometric information such as ocular region, fingerprint, knuckle print, and ear print. In this paper, we discuss the effect of the loss of the finger-selfie details from social media. We demonstrate that this could potentially lead to matching finger-selfies with livescan fingerprints. Further, to prevent the leak of the finger-selfie details, we propose privacy preserving adversarial learning algorithm. The algorithm learns a perturbation to prevent the misuse of finger-selfie towards recognition, yet keeping the visual quality intact to highlight the minuscule object. The experiments are presented on the ISPFDv1 database. Further, we propose a new publicly available Social-Media Posted Finger-selfie (SMPF) Database, containing 1,000 finger-selfie images posted on Instagram.
Improved Ring LWR-Based Key Encapsulation Mechanism Using Cyclotomic Trinomials

In the field of post-quantum cryptography, lattice-based cryptography has received the most noticeable attention. Most lattice-based cryptographic schemes are constructed based on the polynomial ring R <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q</sub> = Z <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">q</sub> [x]/f (x), using a cyclotomic polynomial f (x). Until now, the most preferred cyclotomic polynomials have been x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> + 1, where n is a power of two, and x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> + · · · + x + 1, where n + 1 is a prime. The former results in the smallest decryption error size, but the choice of degree is limited. On the other hand, the latter gives rise to the largest decryption error size, but the choice of degree is very flexible. In this paper, we use a new polynomial ring Rq = Zq/f (x) with a cyclotomic trinomial f (x) = x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n</sup> - x <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">n/2</sup> + 1 as an intermediate that combines the advantages of the other rings. Since the degree n is chosen freely as n = 2 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</sup> 3 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">b</sup> for positive integers a and b, the choice of the degree n is moderate. Furthermore, since the error propagation is small in the middle of polynomial multiplication in the new ring, if the middle part is truncated and used, the decryption error size can be reduced. Based on these observations, we propose a new, practical key encapsulation mechanism (KEM) that is constructed over a ring with a cyclotomic trinomial. The security of our KEM is based on the hardness of ring learning-with-rounding (LWR) problems. With appropriate parameterization for the current 128-bit security model, we show that our KEM obtains shorter secret keys and ciphertexts, especially compared to the previous Ring-LWR-based KEM, Round5, with no error correction code. We then implement our KEM and compare its performance with that of several KEMs that were presented in the second round of the NIST PQC conference.
Deep Learning Model for Human Activity Recognition and Prediction in Smart Homes

To solve the limitation problem of traditional human activity recognition (HAR) tasks which use features extracted manually and some shallow machine learning models, a novel multi-task layer neural network (LSTM) model is proposed based on the ability of deep neural network to automatically extract features in a smart home, combined with the recent successful recurrent neural network and convolution neural network. Prediction techniques based on LSTM neural networks in smart home environments have been used to predicting the next activity as well as on the task of predicting the timestamp of the next event as well. The performance of the model is evaluated on the real dataset. Experimental results show that the LSTM neural networks outperform the other approaches for the prediction of the direct next event, meanwhile, the application of multi-task learning by jointly predicting the next activity and the timestamp of the next event outperforms separate LSTM models for both tasks separately.