Reviewerly/exHarmony huggingface dataset:
FEGAN: Flexible and Efficient Face Editing With Pre-Trained Generator

Since generative adversarial network (GAN) was first proposed, the processing of face images, especially the research of facial attribute editing, has attracted much interest.It not only can alleviate the problems associated with data deficiency, but also has great applications in the field of entertainment.However, existing approaches have limited scalability in the processing of newly-added face attributes, and the quality of generated images is poor.To solve these problems, FEGAN is proposed in this paper to achieve the accurate editing of multi-attribute faces by modifying feature vectors in the latent space.Firstly, a trained generator is used, which greatly reduces the training difficulty of GANs, and the inverse of the generator is used to establish the unique correspondence between the input image and the latent code.Secondly, a linear guide is applied to the latent code, and thus the same distribution as the target image in the latent space is assured.Finally, a generator is used to generate a face image from the guided latent code.The proposed method is utilized for a large number of attribute editing experiments, and the results show that FEGAN can flexibly perform accurate attribute editing while guaranteeing that other areas are not changed.Both qualitative and quantitative results demonstrate its advantages over existing methods.
Inexact stochastic mirror descent for two-stage nonlinear stochastic programs

We introduce an inexact variant of stochastic mirror descent (SMD), called inexact stochastic mirror descent (ISMD), to solve nonlinear two-stage stochastic programs where the second stage problem has linear and nonlinear coupling constraints and a nonlinear objective function which depends on both first and second stage decisions. Given a candidate first stage solution and a realization of the second stage random vector, each iteration of ISMD combines a stochastic subgradient descent using a prox-mapping with the computation of approximate (instead of exact for SMD) primal and dual second stage solutions. We provide two convergence analysis of ISMD, under two sets of assumptions. The first convergence analysis is based on the formulas for inexact cuts of value functions of convex optimization problems shown recently in Guigues (SIAM J. Optim. 30(1), 407–438, 2020). The second convergence analysis provides a convergence rate (the same as SMD) and relies on new formulas that we derive for inexact cuts of value functions of convex optimization problems assuming that the dual function of the second stage problem for all fixed first stage solution and realization of the second stage random vector, is strongly concave. We show that this assumption of strong concavity is satisfied for some classes of problems and present the results of numerical experiments on two simple two-stage problems which show that solving approximately the second stage problem for the first iterations of ISMD can help us obtain a good approximate first stage solution quicker than with SMD.
Improving Candidate Generation for Low-resource Cross-lingual Entity Linking

Cross-lingual entity linking (XEL) is the task of finding referents in a target-language knowledge base (KB) for mentions extracted from source-language texts. The first step of (X)EL is candidate generation, which retrieves a list of plausible candidate entities from the target-language KB for each mention. Approaches based on resources from Wikipedia have proven successful in the realm of relatively high-resource languages, but these do not extend well to low-resource languages with few, if any, Wikipedia pages. Recently, transfer learning methods have been shown to reduce the demand for resources in the low-resource languages by utilizing resources in closely related languages, but the performance still lags far behind their high-resource counterparts. In this paper, we first assess the problems faced by current entity candidate generation methods for low-resource XEL, then propose three improvements that (1) reduce the disconnect between entity mentions and KB entries, and (2) improve the robustness of the model to low-resource scenarios. The methods are simple, but effective: We experiment with our approach on seven XEL datasets and find that they yield an average gain of 16.9% in Top-30 gold candidate recall, compared with state-of-the-art baselines. Our improved model also yields an average gain of 7.9% in in-KB accuracy of end-to-end XEL. 1
Towards Fine-Grained Access Control in Enterprise-Scale Internet-of-Things

Scalable, fine-grained access control for Internet-of-Things is needed in enterprise environments, where tens of thousands of users need to access smart objects which have a similar or larger order of magnitude. Existing solutions offer all-or-nothing access, or require all access to go through a cloud backend, greatly impeding access granularity, robustness and scale. In this paper, we propose Heracles, an IoT access control system which achieves robust, fine-grained access control and responsive execution at enterprise scale. Heracles adopts a capability-based approach using secure, unforgeable tokens that describe the authorizations of users, to either individuals or collections of objects in single or bulk operations. It has a 3-tier architecture to provide centralized policy and distributed execution desired in enterprise environments. Extensive analysis and performance evaluation on a testbed prove that Heracles achieves fine-grained access control and responsive execution at enterprise scale. Compared with systems using access control list, Heracles eliminates or reduces by 10x-100x the updating overhead under frequent changes of subject memberships and policies. Besides, Heracles achieves responsive execution: it takes 0.57 second to access 18 objects which are scattered 1-9 hops away, and execution on a 1-hop or 2-hop object needs only 0.07 or 0.13 second respectively.
A fast conformal predictive system with regularized extreme learning machine

A conformal predictive system(CPS) is based on the learning framework of conformal prediction, which outputs cumulative distribution functions(CDFs) for labels in regression problems. The CDFs output by a CPS provide useful information for users, as they not only provide probability for the events related to the test labels, but also can be transformed to prediction intervals with the corresponding quantiles. Moreover, CPSs have the property of validity since the distributions and intervals they output have statistical compatibility with the realizations. This property is very useful for many risk-sensitive applications such as financial time series forecast and weather forecast. However, as based on conformal predictors, CPSs inherit the computational issue. To build a fast CPS, in this paper, we propose a CPS with regularized extreme learning machine as the underlying algorithm. To be specific, we combine the leave-one-out cross-conformal predictive system(Leave-One-Out CCPS), a variant of the original CPS, with regularized extreme learning machine(RELM), which is named as LOO-CCPS-RELM. We analyse the computational complexity of it and prove its asymptotic validity based on some regularity assumptions. We also prove that the error rate of the prediction interval output by LOO-CCPS-RELM is under control in the asymptotic setting. Experiments with 20 public data sets were conducted to test LOO-CCPS-RELM and the results showed that LOO-CCPS-RELM is empirically valid and compared favourably with the other CPSs.
Improving Code-mixed POS Tagging Using Code-mixed Embeddings

Social media data has become invaluable component of business analytics. A multitude of nuances of social media text make the job of conventional text analytical tools difficult. Code-mixing of text is a phenomenon prevalent among social media users, wherein words used are borrowed from multiple languages, though written in the commonly understood roman script. All the existing supervised learning methods for tasks such as Parts Of Speech (POS) tagging for code-mixed social media (CMSM) text typically depend on a large amount of training data. Preparation of such large training data is resource-intensive, requiring expertise in multiple languages. Though the preparation of small dataset is possible, the out of vocabulary (OOV) words pose major difficulty, while learning models from CMSM text as the number of different ways of writing non-native words in roman script is huge. POS tagging for code-mixed text is non-trivial, as tagging should deal with syntactic rules of multiple languages. The important research question addressed by this article is whether abundantly available unlabeled data can help in resolving the difficulties posed by code-mixed text for POS tagging. We develop an approach for scraping and building word embeddings for code-mixed text illustrating it for Bengali-English, Hindi-English, and Telugu-English code-mixing scenarios. We used a hierarchical deep recurrent neural network with linear-chain CRF layer on top of it to improve the performance of POS tagging in CMSM text by capturing contextual word features and character-sequence–based information. We prepared a labeled resource for POS tagging of CMSM text by correcting 19% of labels from an existing resource. A detailed analysis of the performance of our approach with varying levels of code-mixing is provided. The results indicate that the F1-score of our approach with custom embeddings is better than the CRF-based baseline by 5.81%, 5.69%, and 6.3% in Bengali, Hindi , and Telugu languages, respectively.
Earthquake early warning-enabled smart base isolation system

Recent deployments of Earthquake Early Warning (EEW) system in Japan and some other earthquake-prone regions provide the vital warning signals prior to the arrival of destructive ground motions. The EEW system uses the different traveling speed of seismic P- and S-waves to achieve the goal of earthquake warning. This technology is primarily used to produce warning signals to alert the public to avoid potential risks, such as to evacuate from buildings. It is also used to mitigate other risks such as reducing train speed of Shinkansen trains. The present study suggests a new seismic-risk mitigation technique by connecting a base isolation system with the EEW. Base isolation is a mature technology which decouples structure from its base and lengthens its natural period of vibration. However, existing base isolation devices must possess certain lateral resistance to withstand service lateral forces such as wind, and such stiffness hinders the effectiveness of vibration isolation. In addition, supplementary damping devices are sometimes added to control excessive displacements in isolation level. This paper proposes a smart system which changes the property of a base isolation system upon EEW signal. In normal times when earthquake risk is not present, the base isolation system is locked by shear keys. When an earthquake is signalled by the EEW system, a mechanical system releases the base isolation system. When the earthquake ceases, the system resets and base isolation is locked again. On-board vibration sensors are added to activate the system in case of EEW fails to detect incoming waves. A crucial benefit of the system is that supplementary damper is no longer required to control excessive isolator displacements, and elastic stiffness of base isolation is no longer required to re-enter the base isolation system. The result is that it maximizes the vibration isolation effectiveness. A conceptual framework of proposed system is described and demonstrated by laboratory-scaled experiments. A 6-storey test frame is excited on a shake table subjected to historical earthquakes. Results indicates that the proposed system is effective in reducing earthquake responses on the building. It is an IoT-enabled earthquake-risk mitigation system.
Suggesting Comment Completions for Python using Neural Language Models

Source-code comments are an important communication medium between developers to better understand and maintain software. Current research focuses on auto-generating comments by summarizing the code. However, good comments contain additional details, like important design decisions or required trade-offs, and only developers can decide on the proper comment content. Automated summarization techniques cannot include information that does not exist in the code, therefore fully-automated approaches while helpful, will be of limited use. In our work, we propose to empower developers through a semi-automated system instead. We investigate the feasibility of using neural language models trained on a large corpus of Python documentation strings to generate completion suggestions and obtain promising results. By focusing on confident predictions, we can obtain a top-3 accuracy of over 70%, although this comes at the cost of lower suggestion frequency. Our models can be improved by leveraging context information like the signature and the full body of the method. Additionally, we are able to return good accuracy completions even for new projects, suggesting the generalizability of our approach.
Global context and boundary structure-guided network for cross-modal organ segmentation

In multi-modal medical images such as X-ray and CT, automated organ segmentation is often disturbed by the frequent existence of shape variation, intensity non-uniformities and blurs. Introducing supplementary information helps with eliminating the negative effects of these factors. Medical priors are often leveraged to supplement for specific tasks in previous works, but the specificity of priors hinders these methods from generalizing to cross-modal problems. In this paper we propose Global Context and Boundary Structure-guided Network (GCBSN), which utilizes global context and boundary structure to assist cross-modal organ segmentation. Specifically, we innovatively employ the global context from all spatial regions to guide our deformable convolution. Therefore, more suitable receptive fields can be generated for irregular-shaped targets. Also, we extract the global context contained in each class from the coarse segmentation to assist classifying the areas with non-uniform intensity. Moreover, we design a novel loss that weights more on the errors of positions nearby organ boundaries, and this loss function can avoid errors brought by border blurs. The cross-modal performance of GCBSN is evaluated on two datasets of different modals, i.e., X-Ray images and CT slices. On the 3D NIH Pancreas Dataset, GCBSN outperforms the baseline by 8.72% in terms of the dice coefficient (DC). On the 2D Japanese Society of Radiological Technology Dataset, the mean DC of GCBSN is 98.07% for lung and 94.91% for heart and it surpasses other state-of-the-art methods.
Automatic Data Augmentation Via Deep Reinforcement Learning for Effective Kidney Tumor Segmentation

Conventional data augmentation realized by performing simple pre-processing operations (e.g., rotation, crop, etc.) has been validated for its advantage in enhancing the performance for medical image segmentation. However, the data generated by these conventional augmentation methods are random and sometimes harmful to the subsequent segmentation. In this paper, we developed a novel automatic learning-based data augmentation method for medical image segmentation which models the augmentation task as a trial-and-error procedure using deep reinforcement learning (DRL). In our method, we innovatively combine the data augmentation module and the subsequent segmentation module in an end-to-end training manner with a consistent loss. Specifically, the best sequential combination of different basic operations is automatically learned by directly maximizing the performance improvement (i.e., Dice ratio) on the available validation set. We extensively evaluated our method on CT kidney tumor segmentation which validated the promising results of our method.
Optimal Transport Structure of CycleGAN for Unsupervised Learning for Inverse Problems

Optimal transport (OT) is a mathematical theory that can provide a tool how to transfer one measure to another measure at minimal cost, thus serve another framework for computer vision tasks of image processing without reference. Cycleconsistent generative adversarial network (cycleGAN) is a recent extension of GAN to learn target distributions with less mode collapsing behavior, and also does not need matched data during training. In this article, we explain the link between these two framework by mathematical formula and experimental results. We prove that cycleGAN architecture can be derived from optimal transport problem, and this implies that cycleGAN is a plausible way to learn target distribution when it comes to handling data far from the training set. Using accelerated MR imaging experiments, we confirmed the flexibility and efficacy of our theoretical framework.
Multi-Layer Content Interaction Through Quaternion Product for Visual Question Answering

Multi-modality fusion technologies have greatly improved the performance of neural network-based Video Description/Caption, Visual Question Answering (VQA) and Audio Visual Scene-aware Dialog (AVSD) over the recent years. Most previous approaches only explore the last layers of multiple layer feature fusion while omitting the importance of intermediate layers. To solve the issue for the intermediate layers, we propose an efficient Quaternion Block Network (QBN) to learn interaction not only for the last layer but also for all intermediate layers simultaneously. In our proposed QBN, we use the holistic text features to guide the update of visual features. In the meantime, Hamilton quaternion products can efficiently perform information flow from higher layers to lower layers for both visual and text modalities. The evaluation results show our QBN improve the performance on VQA 2.0, furthermore surpasses the approach using large scale BERT or visual BERT pre-trained models. Extensive ablation study has been carried out to examine the influence of each proposed module in this study.
Weakly Supervised Crowd-Wise Attention For Robust Crowd Counting

Due to a wide range of various application scenes, robust crowd counting is still quite difficult and the performance is far from being satisfied. In this paper, we propose a novel robust crowd counting method by introducing a weakly supervised crowd-wise attention network. The proposed work improves the counting accuracy and robustness by: i) Weakly-supervised crowd segmentation. With a generated segmentation label using motion-guided region-growth, both the appearance feature of one-labeled image and motion features abstracted from its adjacent unlabeled frames, are combined to implement weakly supervised crowd region segmentation, with which active crowd region can be finely perceived from different background disturbances. ii) More accurate spatial attention. We generate a spatial attention map based on the active crowd segmentation, which is used to reweigh the appearance feature to achieve attention-based density estimation. Evaluation of the widely used World Expo' 10 dataset shows that the proposed work can achieve state-of-the-art performance on both accuracy and robustness.
Gaussian Lpcnet for Multisample Speech Synthesis

LPCNet vocoder has recently been presented to TTS community and is now gaining increasing popularity due to its effectiveness and high quality of the speech synthesized with it. In this work, we present a modification of LPCNet that is 1.5x faster, has twice less non-zero parameters and synthesizes speech of the same quality. Such enhancement is possible mostly due to two features that we introduce into the original architecture: the proposed vocoder is designed to generate 16-bit signal instead of 8-bit μ-companded signal, and it predicts two consecutive excitation values at a time independently of each other. To show that these modifications do not lead to quality degradation we train models for five different languages and perform extensive human evaluation.
Path Oblivious Heap: Optimal and Practical Oblivious Priority Queue

We propose Path Oblivious Heap, an extremely simple, practical, and optimal oblivious priority queue.Our construction also implies a practical and optimal oblivious sorting algorithm which we call Path Oblivious Sort.Not only are our algorithms asymptotically optimal, we show that their practical performance is only a small constant factor worse than insecure baselines.More specificially, assuming roughly logarithmic client private storage, Path Oblivious Heap consumes 2× to 7× more bandwidth than the ordinary insecure binary heap; and Path Oblivious Sort consumes 4.5× to 6× more bandwidth than the insecure Merge Sort.We show that these performance results improve existing works by 1-2 orders of magnitude.Finally, we evaluate our algorithm for a multi-party computation scenario and show 7× to 8× reduction in the number of symmetric encryptions relative to the state of the art 1 .
Knowledge Distillation and Random Erasing Data Augmentation for Text-Dependent Speaker Verification

This paper explores the Knowledge Distillation (KD) approach and a data augmentation technique to improve the generalization ability and robustness of text-dependent speaker verification (SV) systems. The KD method consists of two neural networks, known as Teacher and Student, where the student is trained to replicate the predictions from the teacher, so it learns their variability during the training process. To provide robustness to the distillation process, we apply Random Erasing (RE), a data augmentation technique which was created to improve the generalization ability of the neural networks. We have developed two alternatives of the combination of KD and RE, which, produce a more robust system with better performance, since the student network can learn from teacher predictions of data not existing in the original dataset. All alternatives were tested on RSR2015-Part I database, where the proposed variants outperform reference system based on a single network using RE.
SkinAugment: Auto-Encoding Speaker Conversions for Automatic Speech Translation

We propose autoencoding speaker conversion for training data augmentation in automatic speech translation. This technique directly transforms an audio sequence, resulting in audio thesized to resemble another speaker's voice. Our method compares favorably to SpecAugment on English-French and English-Romanian automatic speech translation (AST) tasks as well as on a low-resource English automatic speech recognition (ASR) task. Further, in ablations, we show the benefits of both quantity and diversity in augmented data. Finally, we show that we can combine our approach with augmentation by machine-translated transcripts to obtain a competitive end-to-end AST model that outperforms a very strong cascade model on an English-French AST task. Our method is sufficiently general that it can be applied to other speech generation and analysis tasks.