Reviewerly/exHarmony huggingface dataset:
Image representation of pose-transition feature for 3D skeleton-based action recognition

Recently, skeleton-based human action recognition has received more interest from industrial and research communities for many practical applications thanks to the popularity of depth sensors. A large number of conventional approaches, which have exploited handcrafted features with traditional classifiers, cannot learn high-level spatiotemporal features to precisely recognize complex human actions. In this paper, we introduce a novel encoding technique, namely Pose-Transition Feature to Image (PoT2I), to transform skeleton information to image-based representation for deep convolutional neural networks (CNNs). The spatial joint correlations and temporal pose dynamics of an action are exhaustively depicted by an encoded color image. For learning action models, we fine-tune end-to-end a pre-trained network to thoroughly capture multiple high-level features at multi-scale action representation. The proposed method is benchmarked on several challenging 3D action recognition datasets (e.g., UTKinect-Action3D, SBU-Kinect Interaction, and NTU RGB+D) with different parameter configurations for performance analysis. Outstanding experimental results with the highest accuracy of 90.33% on the most challenging NTU RGB+D dataset demonstrate that our action recognition method with PoT2I outperforms state-of-the-art approaches.
Numerical Method for Comparison on Homomorphically Encrypted Numbers

We propose a new method to compare numbers which are encrypted by Homomorphic Encryption (HE). Previously, comparison and min/max functions were evaluated using Boolean functions where input numbers are encrypted bit-wise. However, the bit-wise encryption methods require relatively expensive computations for basic arithmetic operations such as addition and multiplication. In this paper, we introduce iterative algorithms that approximately compute the min/max and comparison operations of several numbers which are encrypted word-wise. From the concrete error analyses, we show that our min/max and comparison algorithms have $$\varTheta (\alpha )$$ and $$\varTheta (\alpha \log \alpha )$$ computational complexity to obtain approximate values within an error rate $$2^{-\alpha }$$, while the previous minimax polynomial approximation method requires the exponential complexity $$\varTheta (2^{\alpha /2})$$ and $$\varTheta (\sqrt{\alpha }\cdot 2^{\alpha /2})$$, respectively. Our algorithms achieve (quasi-)optimality in terms of asymptotic computational complexity among polynomial approximations for min/max and comparison operations. The comparison algorithm is extended to several applications such as computing the top-k elements and counting numbers over the threshold in encrypted state. Our method enables word-wise HEs to enjoy comparable performance in practice with bit-wise HEs for comparison operations while showing much better performance on polynomial operations. Computing an approximate maximum value of any two $$\ell $$-bit integers encrypted by HEAAN, up to error $$2^{\ell -10}$$, takes only 1.14 ms in amortized running time, which is comparable to the result based on bit-wise HEs.
Learning to Collocate Neural Modules for Image Captioning

We do not speak word by word from scratch; our brain quickly structures a pattern like STH DO STH AT SOMEPLACE and then fills in the detailed descriptions. To render existing encoder-decoder image captioners such humanlike reasoning, we propose a novel framework: learning to Collocate Neural Modules (CNM), to generate the “inner pattern” connecting visual encoder and language decoder. Unlike the widely-used neural module networks in visual Q&A, where the language (i.e., question) is fully observable, CNM for captioning is more challenging as the language is being generated and thus is partially observable. To this end, we make the following technical contributions for CNM training: 1) compact module design - one for function words and three for visual content words (e.g., noun, adjective, and verb), 2) soft module fusion and multistep module execution, robustifying the visual reasoning in partial observation, 3) a linguistic loss for module controller being faithful to part-of-speech collocations (e.g., adjective is before noun). Extensive experiments on the challenging MS-COCO image captioning benchmark validate the effectiveness of our CNM image captioner. In particular, CNM achieves a new state-of-the-art 127.9 CIDErD on Karpathy split and a single-model 126.0 c40 on the official server. CNM is also robust to few training samples, e.g., by training only one sentence per image, CNM can halve the performance loss compared to a strong baseline.
ATMFN: Adaptive-Threshold-Based Multi-Model Fusion Network for Compressed Face Hallucination

Although tremendous strides have been recently made in face hallucination, exiting methods based on a single deep learning framework can hardly satisfactorily provide fine facial features from tiny faces under complex degradation. This article advocates an adaptive-threshold-based multi-model fusion network (ATMFN) for compressed face hallucination, which unifies different deep learning models to take advantages of their respective learning merits. First of all, we construct CNN-, GAN- and RNN-based underlying super-resolvers to produce candidate SR results. Further, the attention subnetwork is proposed to learn the individual fusion weight matrices capturing the most informative components of the candidate SR faces. Particularly, the hyper-parameters of the fusion matrices and the underlying networks are optimized together in an end-to-end manner to drive them for collaborative learning. Finally, a threshold-based fusion and reconstruction module is employed to exploit the candidates' complementarity and thus generate high-quality face images. Extensive experiments on benchmark face datasets and real-world samples show that our model outperforms the state-of-the-art SR methods in terms of quantitative indicators and visual effects. The code and configurations are released at https://github.com/kuihua/ATMFN.
Review of real-time three-dimensional shape measurement techniques

This paper reviews the state-of-the-art real-time three-dimensional shape measurement techniques that are capable of reconstructing the dynamic objects. In this review paper, we only concern the real-time data acquisition ability and the measurement accuracy since the real-time processing and displaying ability relies on the supporting hardware and the adopted fast algorithms instead of the measurement schemes. Based on their working principles, we divide the reviewed real-time three-dimensional shape measurement techniques into three classes: structured light, stereo vision and time of flight. Almost all the reviewed techniques use one-shot measurement scheme and thus not affected by the motion-caused vibration noises. Many measurement techniques have generated quite a lot widely-used commercial products while other measurement techniques still have a long way to go before they become mature for commercial usage. We compare the advantages and disadvantages of these techniques extensively, analyze their limitations and discuss their prospects.
Document-Level Text Classification Using Single-Layer Multisize Filters Convolutional Neural Network

The rapid growth of electronic documents are causing problems like unstructured data that need more time and effort to search a relevant document. Text Document Classification (TDC) has a great significance in information processing and retrieval where unstructured documents are organized into pre-defined classes. Urdu is the most favorite research language in South Asian languages because of its complex morphology, unique features, and lack of linguistic resources like standard datasets. As compared to short text, like sentiment analysis, long text classification needs more time and effort because of large vocabulary, more noise, and redundant information. Machine Learning (ML) and Deep Learning (DL) models have been widely used in text processing. Despite the major limitations of ML models, like learn directed features, these are the favorite methods for Urdu TDC. To the best of our knowledge, it is the first study of Urdu TDC using DL model. In this paper, we design a large multi-purpose and multi-format dataset that contain more than ten thousand documents organize into six classes. We use Single-layer Multisize Filters Convolutional Neural Network (SMFCNN) for classification and compare its performance with sixteen ML baseline models on three imbalanced datasets of various sizes. Further, we analyze the effects of preprocessing methods on SMFCNN performance. SMFCNN outperformed the baseline classifiers and achieved 95.4%, 91.8%, and 93.3% scores of accuracy on medium, large and small size dataset respectively. The designed dataset would be publically and freely available in different formats for future research in Urdu text processing.
An Efficient Reachability-Based Framework for Provably Safe Autonomous Navigation in Unknown Environments

Real-world autonomous vehicles often operate in a priori unknown environments. Since most of these systems are safety-critical, it is important to ensure they operate safely even when faced with environmental uncertainty. Current safety analysis tools enable autonomous systems to reason about safety given full information about the state of the environment a priori. However, these tools do not scale well to scenarios where the environment is being sensed in real time, such as during navigation tasks. In this work, we propose a novel, real-time safety analysis method based on Hamilton-Jacobi reachability that provides strong safety guarantees despite the unknown parts of the environment. Our safety method is planner-agnostic and provides guarantees for a variety of mapping sensors. We demonstrate our approach in simulation and in hardware to provide safety guarantees around a state-of-the-art vision-based, learning-based planner. Videos of our approach and experiments are available on the project website <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
Quality-Aware Feature Aggregation Network for Robust RGBT Tracking

RGBT tracking becomes a popular computer vision task, and has a variety of applications in visual surveillance systems, self-driving cars and intelligent transportation system. This paper investigates how to perform robust visual tracking in adverse and challenging conditions using complementary visual and thermal infrared data (RGBT tracking). We propose a novel deep network architecture called quality-aware Feature Aggregation Network (FANet) for robust RGBT tracking. Unlike existing RGBT trackers, our FANet aggregates hierarchical deep features within each modality to dispose the challenge of significant changes in appearance which is triggered by low illumination,deformation, background clutter and occlusion. In particular, we employ the operations of max pooling to transform these hierarchical and multi-resolution features into uniform space with the same resolution, and use 1×1 convolution operation to compress feature dimensions to achieve more effective hierarchical feature aggregation. To model the interactions between RGB and thermal modalities, we elaborately design an adaptive aggregation subnetwork to integrate features from different modalities based on their reliabilities and thus are able to alleviate noise effects introduced by low-quality sources. The whole FANet is trained in an end-to-end manner. Extensive experiments on large-scale benchmark datasets demonstrate the high-accurate performance against other state-of-the-art RGBT tracking methods.
Blockchain-Based Secure Data Storage for Distributed Vehicular Networks

In this paper, a blockchain-based secure data sharing mechanism is proposed for Vehicular Networks (VNs). Edge service providers are introduced along with ordinary nodes to efficiently manage service provisioning. The edge service providers are placed in the neighborhood of the ordinary nodes to ensure smooth communication between them. The huge amount of data generated by smart vehicles is stored in a distributed file storage system, known as Interplanetary File System (IPFS). It is used to tackle the issues related to data storage in centralized architectures, such as data tampering, lack of privacy, vulnerability to hackers, etc. Monetary incentives are given to edge vehicle nodes to motivate them for accurate and timely service provisioning to ordinary nodes. In response, ordinary nodes give reviews to the edge nodes against the services provided by them, which are further stored in a blockchain to ensure integrity, security and transparency. Smart contracts are used to automate the system processes without the inclusion of an intermediate party and to check the reviews given to the edge nodes. To optimize gas consumption and to enhance the system performance, a Proof of Authority (PoA) consensus mechanism is used to validate the transactions. Moreover, a caching system is introduced at the edge nodes to store frequently used services. Furthermore, both security and privacy are enhanced in the proposed system by incorporating a symmetric key cryptographic mechanism. A trust management mechanism is also proposed in this work to calculate the nodes’ reputation values based upon their trust values. These values determine the authenticity of the nodes involved in the network. Eventually, it is concluded from the simulation results that the proposed system is efficient for VNs.
SwapAdvisor

It is known that deeper and wider neural networks can achieve better accuracy. But it is difficult to continue the trend to increase model size due to limited GPU memory. One promising solution is to support swapping between GPU and CPU memory. However, existing work on swapping only handle certain models and do not achieve satisfactory performance. Deep learning computation is commonly expressed as a dataflow graph which can be analyzed to improve swapping. We propose SwapAdvisor, which performs joint optimization along 3 dimensions based on a given dataflow graph: operator scheduling, memory allocation, and swap decisions. SwapAdvisor explores the vast search space using a custom-designed genetic algorithm. Evaluations using a variety of large models show that SwapAdvisor can train models up to 12 times the GPU memory limit while achieving 53-99% of the throughput of a hypothetical baseline with infinite GPU memory.
Insulator Defect Recognition Based on Global Detection and Local Segmentation

Locating the tiny insulator defect object with complex backgrounds in high-resolution aerial images is a challenging task. In this paper, we propose a novel method which cascades detection and segmentation networks to identify the defect from the global and local two levels: (1) The improved Faster R-CNN is carried out to capture both defects and insulators in the entire image. ResNeXt-101 is adopted as the feature extraction network so as to fully extract features, and Feature Pyramid Network (FPN) is built to enhance the ability of detecting small targets. In addition, the Online Hard Example Mining (OHEM) training strategy is applied to solve the imbalance problem of positive and negative samples. (2) All the detected insulators are extracted and fed into the improved U-Net network to futher inspect at pixel level, we utilize the pre-trained ResNeXt-50 as the encoder of U-Net, incorporate an attention module, Spatial and Channel Squeeze & Excitation Block (SCSE), into the decoding path to highlight the meaningful information. A hybrid loss which merges binary cross entropy (BCE) loss and dice coefficient loss is designed to train our network for figuring out the class imbalance issue. The missed detection can be greatly reduced with the combination of two modified network, which makes comprehensive use of the original map information and local information. On the test set of actual images, the insulator defect recognition precision and recall of the cascade network is 91.9% and 95.7%, exhibiting strong robustness and accuracy.
Financial Defaulter Detection on Online Credit Payment via Multi-view Attributed Heterogeneous Information Network

Default user detection plays one of the backbones in credit risk forecasting and management. It aims at, given a set of corresponding features, e.g., patterns extracted from trading behaviors, predicting the polarity indicating whether a user will fail to make required payments in the future. Recent efforts attempted to incorporate attributed heterogeneous information network (AHIN) for extracting complex interactive features of users and achieved remarkable success on discovering specific default users such as fraud, cash-out users, etc. In this paper, we consider default users, a more general concept in credit risk, and propose a multi-view attributed heterogeneous information network based approach coined MAHINDER to remedy the special challenges. First, multiple views of user behaviors are adopted to learn personal profile due to the endogenous aspect of financial default. Second, local behavioral patterns are specifically modeled since financial default is adversarial and accumulated. With the real datasets contained 1.38 million users on Alibaba platform, we investigate the effectiveness of MAHINDER, and the experimental results exhibit the proposed approach is able to improve AUC over 2.8% and [email protected]=0.1 over 13.1% compared with the state-of-the-art methods. Meanwhile, MAHINDER has as good interpretability as tree-based methods like GBDT, which buoys the deployment in online platforms.
An Empirical Study of Model-Agnostic Techniques for Defect Prediction Models

Software analytics have empowered software organisations to support a wide range of improved decision-making and policy-making. However, such predictions made by software analytics to date have not been explained and justified. Specifically, current defect prediction models still fail to explain why models make such a prediction and fail to uphold the privacy laws in terms of the requirement to explain any decision made by an algorithm. In this paper, we empirically evaluate three model-agnostic techniques, i.e., two state-of-the-art Local Interpretability Model-agnostic Explanations technique (LIME) and BreakDown techniques, and our improvement of LIME with Hyper Parameter Optimisation (LIME-HPO). Through a case study of 32 highly-curated defect datasets that span across 9 open-source software systems, we conclude that (1) model-agnostic techniques are needed to explain individual predictions of defect models; (2) instance explanations generated by model-agnostic techniques are mostly overlapping (but not exactly the same) with the global explanation of defect models and reliable when they are re-generated; (3) model-agnostic techniques take less than a minute to generate instance explanations; and (4) more than half of the practitioners perceive that the contrastive explanations are necessary and useful to understand the predictions of defect models. Since the implementation of the studied model-agnostic techniques is available in both Python and R, we recommend model-agnostic techniques be used in the future.
Transparent Polynomial Delegation and Its Applications to Zero Knowledge Proof

We present a new succinct zero knowledge argument scheme for layered arithmetic circuits without trusted setup. The prover time is O(C + nlogn) and the proof size is O(D logC +log <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> n) for a D-depth circuit with n inputs and C gates. The verification time is also succinct, O(D logC + log <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> n), if the circuit is structured. Our scheme only uses lightweight cryptographic primitives such as collision-resistant hash functions and is plausibly post-quantum secure. We implement a zero knowledge argument system, Virgo, based on our new scheme and compare its performance to existing schemes. Experiments show that it only takes 53 seconds to generate a proof for a circuit computing a Merkle tree with 256 leaves, at least an order of magnitude faster than all other succinct zero knowledge argument schemes. The verification time is 50ms, and the proof size is 253KB, both competitive to existing systems.Underlying Virgo is a new transparent zero knowledge verifiable polynomial delegation scheme with logarithmic proof size and verification time. The scheme is in the interactive oracle proof model and may be of independent interest.
Improved tripartite uncertainty relation with quantum memory

Uncertainty principle is a striking and fundamental feature in quantum mechanics distinguishing from classical mechanics. It offers an important lower bound to predict outcomes of two arbitrary incompatible observables measured on a particle. In quantum information theory, this uncertainty principle is popularly formulized in terms of entropy. Here, we present an improvement of tripartite quantum-memory-assisted entropic uncertainty relation. The uncertainty's lower bound is derived by considering mutual information and Holevo quantity. It shows that the bound derived by this method will be tighter than the lower bound in [Phys. Rev. Lett. 103, 020402 (2009)]. Furthermore, regarding a pair of mutual unbiased bases as the incompatibility, our bound will become extremely tight for the three-qubit $\emph{X}$-state system, completely coinciding with the entropy-based uncertainty, and can restore Renes ${\emph{et al.}}$'s bound with respect to arbitrary tripartite pure states. In addition, by applying our lower bound, one can attain the tighter bound of quantum secret key rate, which is of basic importance to enhance the security of quantum key distribution protocols.
Europarl-ST: A Multilingual Corpus for Speech Translation of Parliamentary Debates

Current research into spoken language translation (SLT), or speech-to-text translation, is often hampered by the lack of specific data resources for this task, as currently available SLT datasets are restricted to a limited set of language pairs. In this paper we present Europarl-ST, a novel multilingual SLT corpus containing paired audio-text samples for SLT from and into 6 European languages, for a total of 30 different translation directions. This corpus has been compiled using the debates held in the European Parliament in the period between 2008 and 2012. This paper describes the corpus creation process and presents a series of automatic speech recognition, machine translation and spoken language translation experiments that highlight the potential of this new resource. The corpus is released under a Creative Commons license and is freely accessible and downloadable.