Miracle-dz/UniFAR-FTUs huggingface dataset:
Large-scale Cloze Test Dataset Designed by Teachers

Cloze test is widely adopted in language exams to evaluate students' language proficiency. In this paper, we propose the first large-scale human-designed cloze test dataset CLOTH in which the questions were used in middle-school and high-school language exams. With the missing blanks carefully created by teachers and candidate choices purposely designed to be confusing, CLOTH requires a deeper language understanding and a wider attention span than previous automatically generated cloze datasets. We show humans outperform dedicated designed baseline models by a significant margin, even when the model is trained on sufficiently large external data. We investigate the source of the performance gap, trace model deficiencies to some distinct properties of CLOTH, and identify the limited ability of comprehending a long-term context to be the key bottleneck. In addition, we find that human-designed data leads to a larger gap between the model's performance and human performance when compared to automatically generated data.
Interpretable Facial Relational Network Using Relational Importance

Human face analysis is an important task in computer vision. According to cognitive-psychological studies, facial dynamics could provide crucial cues for face analysis. In particular, the motion of facial local regions in facial expression is related to the motion of other facial regions. In this paper, a novel deep learning approach which exploits the relations of facial local dynamics has been proposed to estimate facial traits from expression sequence. In order to exploit the relations of facial dynamics in local regions, the proposed network consists of a facial local dynamic feature encoding network and a facial relational network. The facial relational network is designed to be interpretable. Relational importance is automatically encoded and facial traits are estimated by combining relational features based on the relational importance. The relations of facial dynamics for facial trait estimation could be interpreted by using the relational importance. By comparative experiments, the effectiveness of the proposed method has been validated. Experimental results show that the proposed method outperforms the state-of-the-art methods in gender and age estimation.
Inter-Annotator Agreement Networks

This work develops a simple information theoretic framework that captures the dynamic of the inter-annotator agreement process and unifies a wide range of approaches in unsupervised learning. Our model consists of a pair of annotators whose goal is to maximize the mutual information between their annotations. Training the model with standard stochastic gradient descent is challenging, but we find an ablation of the model that admits variational approximation to be empirically effective. We illustrate the strength our framework by achieving new state-of-the-art accuracy on unsupervised part-of-speech tagging, in particular 78.7% on the 45-tag Penn WSJ dataset. We also show clear performance improvement in unsupervised entity typing.
DYAN: A Dynamical Atoms Network for Video Prediction

The ability to anticipate the future is essential when making real time critical decisions, provides valuable information to understand dynamic natural scenes, and can help unsupervised video representation learning. State-of-art video prediction is based on LSTM recursive networks and/or generative adversarial network learning. These are complex architectures that need to learn large numbers of parameters, are potentially hard to train, slow to run, and may produce blurry predictions. In this paper, we introduce DYAN, a novel network with very few parameters and easy to train, which produces accurate, high quality frame predictions, significantly faster than previous approaches. DYAN owes its good qualities to its encoder and decoder, which are designed following concepts from systems identification theory and exploit the dynamics-based invariants of the data. Extensive experiments using several standard video datasets show that DYAN is superior generating frames and that it generalizes well across domains.
Superpixel clustering with deep features for unsupervised road segmentation

Vision-based autonomous driving requires classifying each pixel as corresponding to road or not, which can be addressed using semantic segmentation. Semantic segmentation works well when used with a fully supervised model, but in practice, the required work of creating pixel-wise annotations is very expensive. Although weakly supervised segmentation addresses this issue, most methods are not designed for road segmentation. In this paper, we propose a novel approach to road segmentation that eliminates manual annotation and effectively makes use of road-specific cues. Our method has better performance than other weakly supervised methods and achieves 98% of the performance of a fully supervised method, showing the feasibility of road segmentation for autonomous driving without tedious and costly manual annotation.
Degrees of Relatedness: A Unified Framework for Parametricity, Irrelevance, Ad Hoc Polymorphism, Intersections, Unions and Algebra in Dependent Type Theory

Dependent type theory allows us to write programs and to prove properties about those programs in the same language. However, some properties do not require much proof, as they are evident from a program's implementation, e.g. if a polymorphic program is not ad hoc but relationally parametric, then we get parametricity theorems for free. If we want to safely shortcut proofs by relying on the evident good behaviour of a program, then we need a type-level guarantee that the program is indeed well-behaved. This can be achieved by annotating function types with a modality describing the behaviour of functions. We consider a dependent type system with modalities for relational parametricity, irrelevance (i.e. type-checking time erasability of an argument) and ad hoc polymorphism. The interplay of three modalities and dependent types raises a number of questions. For example: If a term depends on a variable with a given modality, then how should its type depend on it? Are all modalities always applicable, e.g. should we consider parametric functions from the booleans to the naturals? Do we need any further modalities in order to properly reason about these three? We develop a type system, based on a denotational presheaf model, that answers these questions. The core idea is to equip every type with a number of relations --- just equality for small types, but more for larger types --- and to describe function behaviour by saying how functions act on those relations. The system has modality-aware equality judgements (ignoring irrelevant parts) and modality-aware proving operators (for proving free theorems) which are even self-applicable. It also supports sized types, some form of intersection and union types, and parametric quantification over algebraic structures. We prove soundness in a denotational presheaf model.
Practical Network Blocks Design with Q-Learning

Convolutional neural network provides an end-to-end solution to train many computer vision tasks and has gained great successes. However, the design of network architectures usually relies heavily on expert knowledge and is hand-crafted. In this paper, we provide a solution to automatically and efficiently design high performance network architectures. To reduce the search space of network design, we focus on constructing network blocks, which can be stacked to generate the whole network. Blocks are generated through an agent, which is trained with Q-learning to maximize the expected accuracy of the searching blocks on the learning task. Distributed asynchronous framework and early stop strategy are used to accelerate the training process. Our experimental results demonstrate that the network architectures designed by our approach perform competitively compared with hand-crafted state-of-the-art networks. We trained the Q-learning on CIFAR-100, and evaluated on CIFAR10 and ImageNet, the designed block structure achieved 3.60% error on CIFAR-10 and competitive result on ImageNet. The Q-learning process can be efficiently trained only on 32 GPUs in 3 days.
The Retransmission of Rumor-related Tweets: Characteristics of Source and Message

This paper investigates the characteristics of rumor-related tweets that would attract retransmission. Drawing on the uses and gratifications (U & G) and influential users' theories, it proposes a rumor retransmission model which comprises variables associated to the source and the message of the tweets. From a total of 5,885 rumor-related tweets about the death of the founding Prime Minister of Singapore Lee Kuan Yew collected, 800 original tweets without a "RT" prefix were selected for analysis. It was found that the experience and connectivity of the source are correlated to retransmission. The age of the account and number of followers have positive relationships with retransmission, while the number of tweets posted and number of friends have negative relationships. For characteristics of the message, attractiveness and expressions of sense of belonging are positively related to retransmission but the use of directed messages and medium-specific features, namely hashtags and URLs have negative relationships with retransmission. Messages with low level of emotions trigger more retweets than those without emotional expressions and those that are highly emotional. Results could suggest that in inauspicious contexts such as the rumored death of a political figure, Twitter users appear to favor less complex content and seem to be discerning and rational in making retransmission decisions.
Real-Time Pose Estimation for Event Cameras with Stacked Spatial LSTM Networks

We present a new method to estimate the 6DOF pose of the event camera solely based on the event stream. Our method first creates the event image from a list of events that occurs in a very short time interval, then a Stacked Spatial LSTM Network (SP-LSTM) is used to learn and estimate the camera pose. Our SP-LSTM comprises a CNN to learn deep features from the event images and a stack of LSTM to learn spatial dependencies in the image features space. We show that the spatial dependency plays an important role in the pose estimation task and the SP-LSTM can effectively learn that information. The experimental results on the public dataset show that our approach outperforms recent methods by a substantial margin. Overall, our proposed method reduces about 6 times the position error and 3 times the orientation error over the state of the art. The source code and trained models will be released.
A Compare-Propagate Architecture with Alignment Factorization for Natural Language Inference

This paper presents a new deep learning architecture for Natural Language Inference (NLI). Firstly, we introduce a new compare-propagate architecture where alignments pairs are compared and then propagated to upper layers for enhanced representation learning. Secondly, we adopt novel factorization layers for efficient compression of alignment vectors into scalar valued features, which are then be used to augment the base word representations. The design of our approach is aimed to be conceptually simple, compact and yet powerful. We conduct experiments on three popular benchmarks, SNLI, MultiNLI and SciTail, achieving state-of-the-art performance on all. A lightweight parameterization of our model enjoys a $\approx 300\%$ reduction in parameter size compared to the ESIM and DIIN, while maintaining competitive performance. Visual analysis shows that our propagated features are highly interpretable, opening new avenues to explainability in neural NLI models.
Deep Face Feature for Face Alignment and Reconstruction

In this paper, we propose a novel face feature extraction method based on deep learning. Using synthesized multi-view face images, we train a deep face feature (DFF) extractor based on the correlation between projections of a face point on images from different views. A feature vector can be extracted for each pixel of the face image based on the trained DFF model, and it is more effective than general purpose feature descriptors for face-related tasks such as alignment, matching, and reconstruction. Based on the DFF, we develop an effective face alignment method with single or multiple face images as input, which iteratively updates landmarks, pose and 3D shape. Experiments demonstrate that our method can achieve state-of-the-art results for face alignment with a single image, and the alignment can be further improved with multi-view face images.
Machine-Learning-Based Future Received Signal Strength Prediction Using Depth Images for mmWave Communications

This paper discusses a machine-learning (ML)-based future received signal strength (RSS) prediction scheme using depth camera images for millimeter-wave (mmWave) networks. The scheme provides the future RSS prediction of any mmWave links within the camera's view, including links where nodes are not transmitting frames. This enables network controllers to conduct network operations before line-of-sight path blockages degrade the RSS. Using the ML techniques, the prediction scheme automatically learns the relationships between the RSS values and the time-series depth images. We apply a powerful neural-network model, which is capable of learning both spatial and long-term temporal relationships: the Convolutional Neural Network + Convolutional Long Short-Term Memory (CNN+ConvLSTM) network. We generate RSS values and depth images simulating a real-life environment using ray-tracing software and three-dimensional computer graphics (3DCG) software, and then we evaluate the accuracy of the proposed scheme and reveal the impact of camera positions on the prediction accuracy. Moreover, we conduct experiments using commercially available IEEE 802.11ad devices and an RGB-D camera to demonstrate the feasibility of the scheme in a real-life environment. Simulation and experimental results show that the CNN+ConvLSTM network achieves a higher-accuracy in future RSS value prediction compared to other ML models. Moreover, the results confirm that the scheme is feasible for real-time prediction systems and show that the scheme predicts RSS values in 0.5 s future with RMS errors of less than 2.1 dB in the simulations and 3.5 dB in the experiments.
Fine-tuned Language Models for Text Classification

Transfer learning has revolutionized computer vision, but existing approaches in NLP still require task-specific modifications and training from scratch. We propose Fine-tuned Language Models (FitLaM), an effective transfer learning method that can be applied to any task in NLP, and introduce techniques that are key for fine-tuning a state-of-the-art language model. Our method significantly outperforms the state-of-the-art on five text classification tasks, reducing the error by 18-24% on the majority of datasets. We open-source our pretrained models and code to enable adoption by the community.
Fully Distributed Multi-sensor Change-point Detection

Change-point detection has been a fundamental problem in quality control, hazards monitoring and cybersecurity. With the ever-growing complexity of system and enlarging number of sensors, multi-sensor detection algorithms are in real demand. However, most existing detection algorithms consider a centralized approach, meaning that one collects all data at one location, which sometimes may be infeasible for real-time detection due to delay and cost of communication. To address this issue, we propose a fully distributed CUSUM-based detection procedure inspired by the idea of average consensus. The algorithm is fully distributed, i.e., it only requires sensors to communicate with their neighbors and no fusion center is needed. We provide theoretical analysis showing that the performance of the fully distributed scheme can match the centralized algorithms under some mild assumptions. Numerical experiments demonstrate the good performance of the algorithm.
Zero-Shot Style Transfer in Text Using Recurrent Neural Networks

Zero-shot translation is the task of translating between a language pair where no aligned data for the pair is provided during training. In this work we employ a model that creates paraphrases which are written in the style of another existing text. Since we provide the model with no paired examples from the source style to the target style during training, we call this task zero-shot style transfer. Herein, we identify a high-quality source of aligned, stylistically distinct text in Bible versions and use this data to train an encoder/decoder recurrent neural model. We also train a statistical machine translation system, Moses, for comparison. We find that the neural network outperforms Moses on the established BLEU and PINC metrics for evaluating paraphrase quality. This technique can be widely applied due to the broad definition of style which is used. For example, tasks like text simplification can easily be viewed as style transfer. The corpus itself is highly parallel with 33 distinct Bible Versions used, and human-aligned due to the presence of chapter and verse numbers within the text. This makes the data a rich source of study for other natural language tasks.
Effective Dimensionality Reduction for Word Embeddings

Pre-trained word embeddings are used in several downstream applications as well as for constructing representations for sentences, paragraphs and documents. Recently, there has been an emphasis on improving the pretrained word vectors through post-processing algorithms. One improvement area is reducing the dimensionality of word embeddings. Reducing the size of word embeddings can improve their utility in memory constrained devices, benefiting several real world applications. In this work, we present a novel technique that efficiently combines PCA based dimensionality reduction with a recently proposed post-processing algorithm (Mu and Viswanath, 2018), to construct effective word embeddings of lower dimensions. Empirical evaluations on several benchmarks show that our algorithm efficiently reduces the embedding size while achieving similar or (more often) better performance than original embeddings. We have released the source code along with this paper.
Joint IDs Embedding and its Applications in E-commerce

E-commerce has become an important part of our daily lives and there are great challenges due to its dynamic and complex business environment. Many machine intelligence techniques are developed to overcome these challenges. One of the essential elements in those techniques is the representation of data, especially for ID-type data, e.g. item ID, product ID, store ID, brand ID, category ID etc. The classical one-hot encoding suffers sparsity problems due to its high dimension. Moreover, it cannot reflect the relationships among IDs, either homogeneous or heterogeneous ones. In this paper, we propose a novel hierarchical embedding model to jointly learn low-dimensional representations for different types of IDs from the implicit feedback of users. Our approach incorporates the structural information among IDs and embeds all types of IDs into a semantic space. The low-dimensional representations can be effectively extended to many applications including recommendation and forecast etc. We evaluate our approach in several scenarios of "Hema App" and the experimental results validate the effectiveness of our approach.
Synthesizing Audio with Generative Adversarial Networks

While Generative Adversarial Networks (GANs) have seen wide success at the problem of synthesizing realistic images, they have seen little application to the problem of unsupervised audio generation. Unlike for images, a barrier to success is that the best discriminative representations for audio tend to be non-invertible, and thus cannot be used to synthesize listenable outputs. In this paper, we introduce WaveGAN, a first attempt at applying GANs to raw audio synthesis in an unsupervised setting. Our experiments on speech demonstrate that WaveGAN can produce intelligible words from a small vocabulary of human speech, as well as synthesize audio from other domains such as bird vocalizations, drums, and piano. Qualitatively, we find that human judges prefer the generated examples from WaveGAN over those from a method which naively apply GANs on image-like audio feature representations.
MSO Queries on Trees: Enumerating Answers under Updates Using Forest Algebras

We investigate efficient enumeration of answers to MSO-definable queries over trees which are subject to local updates. We exhibit an algorithm that uses an O(n) preprocessing phase and enumerates answers with O(log(n)) delay between them. When the tree is updated, the algorithm can avoid repeating expensive preprocessing and restart the enumeration phase within O(log(n)) time. This improves over previous results that require O(log2(n)) time after updates and have O(log2(n)) delay. Our algorithms and complexity results in the paper are presented in terms of node-selecting tree automata representing the MSO queries. To present our algorithm, we introduce a balancing scheme for parse trees of forest algebra formulas that is of its own interest to lift results from strings to trees.
Learning to Maintain Natural Image Statistics

Maintaining natural image statistics is a crucial factor in restoration and generation of realistic looking images. When training CNNs, photorealism is usually attempted by adversarial training (GAN), that pushes the output images to lie on the manifold of natural images. GANs are very powerful, but not perfect. They are hard to train and the results still often suffer from artifacts. In this paper we propose a complementary approach, whose goal is to train a feed-forward CNN to maintain natural internal statistics. We look explicitly at the distribution of features in an image and train the network to generate images with natural feature distributions. Our approach reduces by orders of magnitude the number of images required for training and achieves state-of-the-art results on both single-image super-resolution, and high-resolution surface normal estimation.
End-to-End Deep HDR Imaging with Large Foreground Motions

This paper proposes the first end-to-end deep framework for high dynamic range (HDR) imaging of dynamic scenes with large-scale foreground motions. In state-of-the-art deep HDR imaging such as [13], the problem is formulated as an image composition problem, by first aligning input images using optical flows which are still error-prone due to occlusion and large motions. In our end-to-end approach, HDR imaging is formulated as an image translation problem and no optical flows are used. Moreover, our simple translation network can automatically hallucinate plausible HDR details in the presence of total occlusion, saturation and under-exposure, which are otherwise almost impossible to recover by conventional optimization approaches. We perform extensive qualitative and quantitative comparisons to show that our end-to-end HDR approach produces excellent results where color artifacts and geometry distortion are significantly reduced compared with existing state-ofthe-art methods.
Learned Iterative Decoding for Lossy Image Compression Systems

For lossy image compression systems, we develop an algorithm called iterative refinement, to improve the decoder's reconstruction compared with standard decoding techniques. Specifically, we propose a recurrent neural network approach for nonlinear, iterative decoding. Our neural decoder, which can work with any encoder, employs self-connected memory units that make use of both causal and non-causal spatial context information to progressively reduce reconstruction error over a fixed number of steps. We experiment with variations of our proposed estimator and obtain as much as a 0.8921 decibel (dB) gain over the standard JPEG algorithm and a 0.5848 dB gain over a state-of-the-art neural compression model.