Reviewerly/exHarmony huggingface dataset:
Ocean FAIR Data Services

Well-founded data management systems are of vital importance for ocean observing systems as they ensure that essential data are not only collected but also retained and made accessible for analysis and application by current and future users. Effective data management requires collaboration across activities including observations, metadata and data assembly, quality assurance and control (QA/QC), and data publication that enables local and interoperable discovery and access, and secure archiving that guarantees long-term preservation. To achieve this, data should be Findable, Accessible, Interoperable, and Reusable (FAIR). Here, we outline how these principles apply to ocean data, and illustrate them with a few examples. In recent decades, ocean data managers, in close collaboration with international organizations, have played an active role in the improvement of environmental data standardization, accessibility and interoperability through different projects, enhancing access to observation data at all stages of the data life cycle and fostering the development of integrated services targeted to research, regulatory and operational users. As ocean observing systems evolve and an increasing number of autonomous platforms and sensors are deployed, the volume and variety of data increases dramatically. For instance, there are more than 70 data catalogues that contain metadata records for the polar oceans, a situation that makes comprehensive data discovery beyond the capacity of most researchers. To better serve research, operational, and commercial users, more efficient turnaround of quality data in known formats and made available through web services is necessary. In particular, automation of data workflows will be critical to reduce friction throughout the data value chain. Adhering to the FAIR principles with free, timely and unrestricted access to ocean observation data is beneficial for the originators, has obvious benefits for users and is an essential foundation for the development of new services made possible with big data technologies.
MFC-GAN: Class-imbalanced dataset classification using Multiple Fake Class Generative Adversarial Network

Class-imbalanced datasets are common across different domains such as health, banking, security and others. With such datasets, the learning algorithms are often biased toward the majority class-instances. Data augmentation is a common approach that aims at rebalancing a dataset by injecting more data samples of the minority class instances. In this paper, a new data augmentation approach is proposed using a Generative Adversarial Networks (GAN) to handle the class imbalance problem. Unlike common GAN models, which use a single fake class, the proposed method uses multiple fake classes to ensure a fine-grained generation and classification of the minority class instances. Moreover, the proposed GAN model is conditioned to generate minority class instances aiming at rebalancing the dataset. Extensive experiments were carried out using public datasets, where synthetic samples generated using our model were added to the imbalanced dataset, followed by performing classification using Convolutional Neural Network. Experiment results show that our model can generate diverse minority class instances, even in extreme cases where the number of minority class instances is relatively low. Additionally, superior performance of our model over other common augmentation and oversampling methods was achieved in terms of classification accuracy and quality of the generated samples.
LSTA: Long Short-Term Attention for Egocentric Action Recognition

Egocentric activity recognition is one of the most challenging tasks in video analysis.It requires a fine-grained discrimination of small objects and their manipulation.While some methods base on strong supervision and attention mechanisms, they are either annotation consuming or do not take spatio-temporal patterns into account.In this paper we propose LSTA as a mechanism to focus on features from relevant spatial parts while attention is being tracked smoothly across the video sequence.We demonstrate the effectiveness of LSTA on egocentric activity recognition with an end-to-end trainable two-stream architecture, achieving state-of-the-art performance on four standard benchmarks.
PassGAN: A Deep Learning Approach for Password Guessing

State-of-the-art password guessing tools, such as HashCat and John the Ripper, enable users to check billions of passwords per second against password hashes. In addition to performing straightforward dictionary attacks, these tools can expand password dictionaries using password generation rules, such as concatenation of words (e.g., “password123456”) and leet speak (e.g., “password” becomes “p4s5w0rd”). Although these rules work well in practice, creating and expanding them to model further passwords is a labor-intensive task that requires specialized expertise. To address this issue, in this paper we introduce PassGAN, a novel approach that replaces human-generated password rules with theory-grounded machine learning algorithms. Instead of relying on manual password analysis, PassGAN uses a Generative Adversarial Network (GAN) to autonomously learn the distribution of real passwords from actual password leaks, and to generate high-quality password guesses. Our experiments show that this approach is very promising. When we evaluated PassGAN on two large password datasets, we were able to surpass rule-based and state-of-the-art machine learning password guessing tools. However, in contrast with the other tools, PassGAN achieved this result without any a-priori knowledge on passwords or common password structures. Additionally, when we combined the output of PassGAN with the output of HashCat, we were able to match 51%–73% more passwords than with HashCat alone. This is remarkable, because it shows that PassGAN can autonomously extract a considerable number of password properties that current state-of-the art rules do not encode.
StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning

Real-time strategy games have been an important field of game artificial intelligence in recent years. This paper presents a reinforcement learning and curriculum transfer learning method to control multiple units in StarCraft micro management. We define an efficient state representation, which breaks down the complexity caused by the large state space in the game environment. Then, a parameter sharing multi-agent gradient descent Sarsa(λ) algorithm is proposed to train the units. The learning policy is shared among our units to encourage cooperative behaviors. We use a neural network as a function approximator to estimate the action-value function, and propose a reward function to help units balance their move and attack. In addition, a transfer learning method is used to extend our model to more difficult scenarios, which accelerates the training process and improves the learning performance. In small-scale scenarios, our units successfully learn to combat and defeat the built-in AI with 100% win rates. In large-scale scenarios, the curriculum transfer learning method is used to progressively train a group of units, and it shows superior performance over some baseline methods in target scenarios. With reinforcement learning and curriculum transfer learning, our units are able to learn appropriate strategies in StarCraft micro management scenarios.
Nonlinear optical multi-image encryption scheme with two-dimensional linear canonical transform

A nonlinear optical multi-image encryption scheme based on chaotic system and two-dimensional linear canonical transform (2D LCT) is designed. Firstly, the low-frequency sub-bands of the four grayscale images are extracted with the contourlet transform (CT), respectively. Then the four sub-bands are synthesized into a one-dimensional (1D) sequence, which is scrambled with a chaotic sequence produced by the two-dimensional (2D) logistic map. Subsequently, the image converted from the resulting 1D sequence is re-encrypted into a white noise-like image by phase truncation and phase reservation in the 2D LCT domain. Finally, the XOR operation based on the 2D logistic map is performed on the white noise-like image to yield the final ciphertext image. The parameters of the 2D LCT and the logistic map as the main keys enlarge the key space of the proposed multi-image encryption scheme. In addition, the keys regarded as the initial values of the chaotic systems are related to the original images. The phase truncation and the bitwise XOR operation, as nonlinear processes, improve the robustness of the presented multi-image encryption scheme against chosen-plaintext attack. Numerical simulation results demonstrate the validity and the reliability of the proposed optical multi-image encryption scheme with good performance.
InstaBoost: Boosting Instance Segmentation via Probability Map Guided Copy-Pasting

Instance segmentation requires a large number of training samples to achieve satisfactory performance and benefits from proper data augmentation. To enlarge the training set and increase the diversity, previous methods have investigated using data annotation from other domain (e.g. bbox, point) in a weakly supervised mechanism. In this paper, we present a simple, efficient and effective method to augment the training set using the existing instance mask annotations. Exploiting the pixel redundancy of the background, we are able to improve the performance of Mask R-CNN for 1.7 mAP on COCO dataset and 3.3 mAP on Pascal VOC dataset by simply introducing random jittering to objects. Furthermore, we propose a location probability map based approach to explore the feasible locations that objects can be placed based on local appearance similarity. With the guidance of such map, we boost the performance of R101-Mask R-CNN on instance segmentation from 35.7 mAP to 37.9 mAP without modifying the backbone or network structure. Our method is simple to implement and does not increase the computational complexity. It can be integrated into the training pipeline of any instance segmentation model without affecting the training and inference efficiency. Our code and models have been released at https://github.com/GothicAi/InstaBoost.
FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents

We present a new dataset for form understanding in noisy scanned documents (FUNSD) that aims at extracting and structuring the textual content of forms. The dataset comprises 199 real, fully annotated, scanned forms. The documents are noisy and vary widely in appearance, making form understanding (FoUn) a challenging task. The proposed dataset can be used for various tasks, including text detection, optical character recognition, spatial layout analysis, and entity labeling/linking. To the best of our knowledge, this is the first publicly available dataset with comprehensive annotations to address FoUn task. We also present a set of baselines and introduce metrics to evaluate performance on the FUNSD dataset, which can be downloaded at https://guillaumejaume.github.io/FUNSD.
What Would You Expect? Anticipating Egocentric Actions With Rolling-Unrolling LSTMs and Modality Attention

Egocentric action anticipation consists in understanding which objects the camera wearer will interact with in the near future and which actions they will perform. We tackle the problem proposing an architecture able to anticipate actions at multiple temporal scales using two LSTMs to 1) summarize the past, and 2) formulate predictions about the future. The input video is processed considering three complimentary modalities: appearance (RGB), motion (optical flow) and objects (object-based features). Modality-specific predictions are fused using a novel Modality ATTention (MATT) mechanism which learns to weigh modalities in an adaptive fashion. Extensive evaluations on two large-scale benchmark datasets show that our method outperforms prior art by up to +7% on the challenging EPIC-Kitchens dataset including more than 2500 actions, and generalizes to EGTEA Gaze+. Our approach is also shown to generalize to the tasks of early action recognition and action recognition. Our method is ranked first in the public leaderboard of the EPIC-Kitchens egocentric action anticipation challenge 2019. Please see the project web page for code and additional details: http://iplab.dmi.unict.it/rulstm.
Bilinear Attention Networks for Person Retrieval

This paper investigates a novel Bilinear attention (Bi-attention) block, which discovers and uses second order statistical information in an input feature map, for the purpose of person retrieval. The Bi-attention block uses bilinear pooling to model the local pairwise feature interactions along each channel, while preserving the spatial structural information. We propose an Attention in Attention (AiA) mechanism to build inter-dependency among the second order local and global features with the intent to make better use of, or pay more attention to, such higher order statistical relationships. The proposed network, equipped with the proposed Bi-attention is referred to as Bilinear ATtention network (BAT-net). Our approach outperforms current state-of-the-art by a considerable margin across the standard benchmark datasets (e.g., CUHK03, Market-1501, DukeMTMC-reID and MSMT17).
Artificial Intelligence-Based Mitosis Detection in Breast Cancer Histopathology Images Using Faster R-CNN and Deep CNNs

Breast cancer is the leading cause of mortality in women. Early diagnosis of breast cancer can reduce the mortality rate. In the diagnosis, the mitotic cell count is an important biomarker for predicting the aggressiveness, prognosis, and grade of breast cancer. In general, pathologists manually examine histopathology images under high-resolution microscopes for the detection of mitotic cells. However, because of the minute differences between the mitotic and normal cells, this process is tiresome, time-consuming, and subjective. To overcome these challenges, artificial-intelligence-based (AI-based) techniques have been developed which automatically detect mitotic cells in the histopathology images. Such AI techniques accelerate the diagnosis and can be used as a second-opinion system for a medical doctor. Previously, conventional image-processing techniques were used for the detection of mitotic cells, which have low accuracy and high computational cost. Therefore, a number of deep-learning techniques that demonstrate outstanding performance and low computational cost were recently developed; however, they still require improvement in terms of accuracy and reliability. Therefore, we present a multistage mitotic-cell-detection method based on Faster region convolutional neural network (Faster R-CNN) and deep CNNs. Two open datasets (international conference on pattern recognition (ICPR) 2012 and ICPR 2014 (MITOS-ATYPIA-14)) of breast cancer histopathology were used in our experiments. The experimental results showed that our method achieves the state-of-the-art results of 0.876 precision, 0.841 recall, and 0.858 F1-measure for the ICPR 2012 dataset, and 0.848 precision, 0.583 recall, and 0.691 F1-measure for the ICPR 2014 dataset, which were higher than those obtained using previous methods. Moreover, we tested the generalization capability of our technique by testing on the tumor proliferation assessment challenge 2016 (TUPAC16) dataset and found that our technique also performs well in a cross-dataset experiment which proved the generalization capability of our proposed technique.
Toward Moderate Overparameterization: Global Convergence Guarantees for Training Shallow Neural Networks

Many modern neural network architectures are trained in an overparameterized regime where the parameters of the model exceed the size of the training dataset. Sufficiently overparameterized neural network architectures in principle have the capacity to fit any set of labels including random noise. However, given the highly nonconvex nature of the training landscape it is not clear what level and kind of overparameterization is required for first order methods to converge to a global optima that perfectly interpolate any labels. A number of recent theoretical works have shown that for very wide neural networks where the number of hidden units is polynomially large in the size of the training data gradient descent starting from a random initialization does indeed converge to a global optima. However, in practice much more moderate levels of overparameterization seems to be sufficient and in many cases overparameterized models seem to perfectly interpolate the training data as soon as the number of parameters exceed the size of the training data by a constant factor. Thus there is a huge gap between the existing theoretical literature and practical experiments. In this paper we take a step towards closing this gap. Focusing on shallow neural nets and smooth activations, we show that (stochastic) gradient descent when initialized at random converges at a geometric rate to a nearby global optima as soon as the square-root of the number of network parameters exceeds the size of the training data. Our results also benefit from a fast convergence rate and continue to hold for non-differentiable activations such as Rectified Linear Units (ReLUs).
DSGN: Deep Stereo Geometry Network for 3D Object Detection

Most state-of-the-art 3D object detectors rely heavily on LiDAR sensors and there remains a large gap in terms of performance between image-based and LiDAR-based methods, caused by inappropriate representation for the prediction in 3D scenarios. Our method, called Deep Stereo Geometry Network (DSGN), reduces this gap significantly by detecting 3D objects on a differentiable volumetric representation -- 3D geometric volume, which effectively encodes 3D geometric structure for 3D regular space. With this representation, we learn depth information and semantic cues simultaneously. For the first time, we provide a simple and effective one-stage stereo-based 3D detection pipeline that jointly estimates the depth and detects 3D objects in an end-to-end learning manner. Our approach outperforms previous stereo-based 3D detectors (about 10 higher in terms of AP) and even achieves comparable performance with a few LiDAR-based methods on the KITTI 3D object detection leaderboard. Code will be made publicly available at https://github.com/chenyilun95/DSGN.
Fine-Grained Generalized Zero-Shot Learning via Dense Attribute-Based Attention

We address the problem of fine-grained generalized zero-shot recognition of visually similar classes without training images for some classes. We propose a dense attribute-based attention mechanism that for each attribute focuses on the most relevant image regions, obtaining attribute-based features. Instead of aligning a global feature vector of an image with its associated class semantic vector, we propose an attribute embedding technique that aligns each attribute-based feature with its attribute semantic vector. Hence, we compute a vector of attribute scores, for the presence of each attribute in an image, whose similarity with the true class semantic vector is maximized. Moreover, we adjust each attribute score using an attention mechanism over attributes to better capture the discriminative power of different attributes. To tackle the challenge of bias towards seen classes during testing, we propose a new self-calibration loss that adjusts the probability of unseen classes to account for the training bias. We conduct experiments on three popular datasets of CUB, SUN and AWA2 as well as the large-scale DeepFashion dataset, showing that our model significantly improves the state of the art.
Blockchain-based privacy-preserving remote data integrity checking scheme for IoT information systems

Remote data integrity checking is of great importance to the security of cloud-based information systems. Previous works generally assume a trusted third party to oversee the integrity of the outsourced data, which may be invalid in practice. In this paper, we utilize the blockchain to construct a novel privacy-preserving remote data integrity checking scheme for Internet of Things (IoT) information management systems without involving trusted third parties. Our scheme leverages the Lifted EC-ElGamal cryptosystem, bilinear pairing, and blockchain to support efficient public batch signature verifications and protect the security and data privacy of the IoT systems. The results of the experiment demonstrate the efficiency of our scheme.
An efficient approach for encrypting double color images into a visually meaningful cipher image using 2D compressive sensing

An efficient visually meaningful double color image encryption algorithm is proposed by combining 2D compressive sensing (CS) with an embedding technique. First, two color images are measured by measurement matrices in two directions to achieve simultaneous compression and encryption, in which low-dimensional matrices generated from Logistic-Sine system (LSS) are extended with Kronecker product (KP), and the resulting high-dimensional matrices optimized by singular value decomposition (SVD) are employed as measurement matrices. Second, the compressed cipher images are confused by index sequences produced by a 6D hyperchaotic system. Finally, a visually meaningful cipher image is obtained by embedding permutated cipher images into a color carrier image. The final cipher image and plain image are of the same size, which greatly reduces the storage space and transmission bandwidth. To enhance the relationship of our algorithm with plain images and prevent vulnerability to known-plaintext and chosen-plaintext attacks, SHA 256 hash values and feature parameters of plain images are combined to generate the initial values of the LSS and 6D hyperchaotic system, and these parameters are both embedded into the carrier image to avoid additional transmission and storage. Simulation results and performance analyses demonstrate the effectiveness and security of the proposed image encryption scheme.
Latency performance modeling and analysis for hyperledger fabric blockchain network

Blockchain has been one of the most attractive technologies for many modern and even future applications. Fabric, an open-source framework to implement the permissioned enterprise-grade blockchain, is getting increasing attention from innovators. The latency performance is crucial to the Fabric blockchain in assessing its effectiveness. Many empirical studies were conducted to analyze this performance based on different hardware platforms. These experimental results are not comparable as they are highly dependent on the underlying networks. Moreover, theoretical analysis on the latency of Fabric blockchain still receives much less attention. This paper provides a novel theoretical model to calculate the transaction latency under various network configurations such as block size, block interval, etc. Subsequently, we validate the proposed latency model with experiments, and the results show that the difference between analytical and experimental results is as low as 6.1%. We also identify some performance bottlenecks and give insights from the developer’s perspective.
LightFace: A Hybrid Deep Face Recognition Framework

Face recognition constitutes a relatively a popular area which has emerged from the rulers of the social media to top universities in the world. Those frontiers and rule makers recently designed deep learning based custom face recognition models. A modern face recognition pipeline consists of four common stages: detecting, alignment, representation and verification. However, face recognition studies mainly mention the representation stage of a pipeline. In this paper, first of all a review face recognition has been done and then the description of the developed lightweight hybrid high performance face recognition framework has been made. Its hybrid feature enables to switch face recognition models among state-of-the-art ones.