OpenRaiser/Intern-Atlas huggingface dataset:
Optimizing Filter Size in Convolutional Neural Networks for Facial Action Unit Recognition
Venue: arXiv cs.CV 2017
Year: 2017

Recognizing facial action units (AUs) during spontaneous facial displays is a challenging problem. Most recently, Convolutional Neural Networks (CNNs) have shown promise for facial AU recognition, where predefined and fixed convolution filter sizes are employed. In order to achieve the best performance, the optimal filter size is often empirically found by conducting extensive experimental validation. Such a training process suffers from expensive training cost, especially as the network becomes deeper. This paper proposes a novel Optimized Filter Size CNN (OFS-CNN), where the filter sizes and weights of all convolutional layers are learned simultaneously from the training data along with learning convolution filters. Specifically, the filter size is defined as a continuous variable, which is optimized by minimizing the training loss. Experimental results on two AU-coded spontaneous databases have shown that the proposed OFS-CNN is capable of estimating optimal filter size for varying image resolution and outperforms traditional CNNs with the best filter size obtained by exhaustive search. The OFS-CNN also beats the CNN using multiple filter sizes and more importantly, is much more efficient during testing with the proposed forward-backward propagation algorithm.
Learning a Target Sample Re-Generator for Cross-Database Micro-Expression Recognition
Venue: arXiv cs.CV 2017
Year: 2017

In this paper, we investigate the cross-database micro-expression recognition problem, where the training and testing samples are from two different micro-expression databases. Under this setting, the training and testing samples would have different feature distributions and hence the performance of most existing micro-expression recognition methods may decrease greatly. To solve this problem, we propose a simple yet effective method called Target Sample Re-Generator (TSRG) in this paper. By using TSRG, we are able to re-generate the samples from target micro-expression database and the re-generated target samples would share same or similar feature distributions with the original source samples. For this reason, we can then use the classifier learned based on the labeled source samples to accurately predict the micro-expression categories of the unlabeled target samples. To evaluate the performance of the proposed TSRG method, extensive cross-database micro-expression recognition experiments designed based on SMIC and CASME II databases are conducted. Compared with recent state-of-the-art cross-database emotion recognition methods, the proposed TSRG achieves more promising results.
Anytime Exact Belief Propagation
Venue: arXiv cs.AI 2017
Year: 2017

Statistical Relational Models and, more recently, Probabilistic Programming, have been making strides towards an integration of logic and probabilistic reasoning. A natural expectation for this project is that a probabilistic logic reasoning algorithm reduces to a logic reasoning algorithm when provided a model that only involves 0-1 probabilities, exhibiting all the advantages of logic reasoning such as short-circuiting, intelligibility, and the ability to provide proof trees for a query answer. In fact, we can take this further and require that these characteristics be present even for probabilistic models with probabilities \emph{near} 0 and 1, with graceful degradation as the model becomes more uncertain. We also seek inference that has amortized constant time complexity on a model's size (even if still exponential in the induced width of a more directly relevant portion of it) so that it can be applied to huge knowledge bases of which only a relatively small portion is relevant to typical queries. We believe that, among the probabilistic reasoning algorithms, Belief Propagation is the most similar to logic reasoning: messages are propagated among neighboring variables, and the paths of message-passing are similar to proof trees. However, Belief Propagation is either only applicable to tree models, or approximate (and without guarantees) for precision and convergence. In this paper we present work in progress on an Anytime Exact Belief Propagation algorithm that is very similar to Belief Propagation but is exact even for graphical models with cycles, while exhibiting soft short-circuiting, amortized constant time complexity in the model size, and which can provide probabilistic proof trees.
Example Setups of Navier-Stokes Equations with Control and Observation: Spatial Discretization and Representation via Linear-quadratic Matrix Coefficients
Venue: arXiv cs.MS 2017
Year: 2017

We provide spatial discretizations of nonlinear incompressible Navier-Stokes equations with inputs and outputs in the form of matrices ready to use in any numerical linear algebra package. We discuss the assembling of the system operators and the realization of boundary conditions and inputs and outputs. We describe the two benchmark problems - the driven cavity and the cylinder wake - and provide the corresponding data. The use of the data is illustrated by numerous example setups. The test cases are provided as plain PYTHON or OCTAVE/MATLAB script files for immediate replication.
Exploiting Web Images for Weakly Supervised Object Detection
Venue: arXiv cs.CV 2017
Year: 2017

In recent years, the performance of object detection has advanced significantly with the evolving deep convolutional neural networks. However, the state-of-the-art object detection methods still rely on accurate bounding box annotations that require extensive human labelling. Object detection without bounding box annotations, i.e, weakly supervised detection methods, are still lagging far behind. As weakly supervised detection only uses image level labels and does not require the ground truth of bounding box location and label of each object in an image, it is generally very difficult to distill knowledge of the actual appearances of objects. Inspired by curriculum learning, this paper proposes an easy-to-hard knowledge transfer scheme that incorporates easy web images to provide prior knowledge of object appearance as a good starting point. While exploiting large-scale free web imagery, we introduce a sophisticated labour free method to construct a web dataset with good diversity in object appearance. After that, semantic relevance and distribution relevance are introduced and utilized in the proposed curriculum training scheme. Our end-to-end learning with the constructed web data achieves remarkable improvement across most object classes especially for the classes that are often considered hard in other works.
What Drives People's Choices in Turn-Taking Games, if not Game-Theoretic Rationality?
Venue: arXiv cs.GT 2017
Year: 2017

In an earlier experiment, participants played a perfect information game against a computer, which was programmed to deviate often from its backward induction strategy right at the beginning of the game. Participants knew that in each game, the computer was nevertheless optimizing against some belief about the participant's future strategy. In the aggregate, it appeared that participants applied forward induction. However, cardinal effects seemed to play a role as well: a number of participants might have been trying to maximize expected utility. In order to find out how people really reason in such a game, we designed centipede-like turn-taking games with new payoff structures in order to make such cardinal effects less likely. We ran a new experiment with 50 participants, based on marble drop visualizations of these revised payoff structures. After participants played 48 test games, we asked a number of questions to gauge the participants' reasoning about their own and the opponent's strategy at all decision nodes of a sample game. We also checked how the verbalized strategies fit to the actual choices they made at all their decision points in the 48 test games. Even though in the aggregate, participants in the new experiment still tend to slightly favor the forward induction choice at their first decision node, their verbalized strategies most often depend on their own attitudes towards risk and those they assign to the computer opponent, sometimes in addition to considerations about cooperativeness and competitiveness.
Categories for Dynamic Epistemic Logic
Venue: arXiv cs.LO 2017
Year: 2017

The primary goal of this paper is to recast the semantics of modal logic, and dynamic epistemic logic (DEL) in particular, in category-theoretic terms. We first review the category of relations and categories of Kripke frames, with particular emphasis on the duality between relations and adjoint homomorphisms. Using these categories, we then reformulate the semantics of DEL in a more categorical and algebraic form. Several virtues of the new formulation will be demonstrated: The DEL idea of updating a model into another is captured naturally by the categorical perspective -- which emphasizes a family of objects and structural relationships among them, as opposed to a single object and structure on it. Also, the categorical semantics of DEL can be merged straightforwardly with a standard categorical semantics for first-order logic, providing a semantics for first-order DEL.
Self-confirming Games: Unawareness, Discovery, and Equilibrium
Venue: arXiv cs.GT 2017
Year: 2017

Equilibrium notions for games with unawareness in the literature cannot be interpreted as steady-states of a learning process because players may discover novel actions during play. In this sense, many games with unawareness are "self-destroying" as a player's representation of the game must change after playing it once. We define discovery processes where at each state there is an extensive-form game with unawareness that together with the players' play determines the transition to possibly another extensive-form games with unawareness in which players are now aware of actions that they have previously discovered. A discovery process is rationalizable if players play extensive-form rationalizable strategies in each game with unawareness. We show that for any game with unawareness there is a rationalizable discovery process that leads to a self-confirming game that possesses an extensive-form rationalizable self-confirming equilibrium. This notion of equilibrium can be interpreted as steady-state of a learning and discovery process.
An Evolutionary Stochastic-Local-Search Framework for One-Dimensional Cutting-Stock Problems
Venue: arXiv cs.NE 2017
Year: 2017

We introduce an evolutionary stochastic-local-search (SLS) algorithm for addressing a generalized version of the so-called 1/V/D/R cutting-stock problem. Cutting-stock problems are encountered often in industrial environments and the ability to address them efficiently usually results in large economic benefits. Traditionally linear-programming-based techniques have been utilized to address such problems, however their flexibility might be limited when nonlinear constraints and objective functions are introduced. To this end, this paper proposes an evolutionary SLS algorithm for addressing one-dimensional cutting-stock problems. The contribution lies in the introduction of a flexible structural framework of the optimization that may accommodate a large family of diversification strategies including a novel parallel pattern appropriate for SLS algorithms (not necessarily restricted to cutting-stock problems). We finally demonstrate through experiments in a real-world manufacturing problem the benefit in cost reduction of the considered diversification strategies.
Impact of Correlation between Interferers on Coverage Probability and rate in Cellular Systems
Venue: arXiv cs.IT 2017
Year: 2017

When the user channel experiences Nakagami-m fading, the coverage probability expressions are theoretically compared for the following cases: (i). The N interferers are independent $η$-$μ$ random variables (RVs). (ii). The N interferers are correlated $η$-$μ$ RVs. It is analytically shown that the coverage probability in the presence of correlated interferers is greater than or equal to the coverage probability in the presence of independent interferers when the shape parameter of the channel between the user and its base station (BS) is not greater than one. Further, rate is compared for the following cases: (i). The user channel experiences $η$-$μ$ RV and the $N$ interferers are independent $η$-$μ$ RVs. (ii). The N interferers are correlated $η$-$μ$ RVs. It is analytically shown that the rate in the presence of correlated interferers is greater than or equal to the rate in the presence of independent interferers. Simulation results are provided and these match with the obtained theoretical results. The utility of our results are also discussed.
Nearest Common Ancestors: Universal Trees and Improved Labeling Schemes
Venue: arXiv cs.DS 2017
Year: 2017

We investigate the nearest common ancestor (NCA) function in rooted trees. As the main conceptual contribution, the paper introduces universal trees for the NCA function: For a given family of rooted trees, an NCA-universal tree $S$ is a rooted tree such that any tree $T$ of the family can be embedded into $S$ such that the embedding of the NCA in $T$ of two nodes of $T$ is equal to the NCA in $S$ of the embeddings of the two nodes. As the main technical result we give explicit constructions of NCA-universal trees of size $n^{2.318}$ for the family of rooted $n$-vertex trees and of size $n^{1.894}$ for the family of rooted binary $n$-vertex trees. A direct consequence is the explicit construction of NCA-labeling schemes with labels of size $2.318\log_2 n$ and $1.894\log_2 n$ for the two families of rooted trees. This improves on the best known such labeling schemes established by Alstrup, Halvorsen and Larsen [SODA 2014].
A Comparative Study of the Clinical use of Motion Analysis from Kinect Skeleton Data
Venue: arXiv cs.CV 2017
Year: 2017

The analysis of human motion as a clinical tool can bring many benefits such as the early detection of disease and the monitoring of recovery, so in turn helping people to lead independent lives. However, it is currently under used. Developments in depth cameras, such as Kinect, have opened up the use of motion analysis in settings such as GP surgeries, care homes and private homes. To provide an insight into the use of Kinect in the healthcare domain, we present a review of the current state of the art. We then propose a method that can represent human motions from time-series data of arbitrary length, as a single vector. Finally, we demonstrate the utility of this method by extracting a set of clinically significant features and using them to detect the age related changes in the motions of a set of 54 individuals, with a high degree of certainty (F1- score between 0.9 - 1.0). Indicating its potential application in the detection of a range of age-related motion impairments.
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
Venue: arXiv cs.CV 2017
Year: 2017

The original ImageNet dataset is a popular large-scale benchmark for training Deep Neural Networks. Since the cost of performing experiments (e.g, algorithm design, architecture search, and hyperparameter tuning) on the original dataset might be prohibitive, we propose to consider a downsampled version of ImageNet. In contrast to the CIFAR datasets and earlier downsampled versions of ImageNet, our proposed ImageNet32$\times$32 (and its variants ImageNet64$\times$64 and ImageNet16$\times$16) contains exactly the same number of classes and images as ImageNet, with the only difference that the images are downsampled to 32$\times$32 pixels per image (64$\times$64 and 16$\times$16 pixels for the variants, respectively). Experiments on these downsampled variants are dramatically faster than on the original ImageNet and the characteristics of the downsampled datasets with respect to optimal hyperparameters appear to remain similar. The proposed datasets and scripts to reproduce our results are available at http://image-net.org/download-images and https://github.com/PatrykChrabaszcz/Imagenet32_Scripts
Expressive Stream Reasoning with Laser
Venue: arXiv cs.LO 2017
Year: 2017

An increasing number of use cases require a timely extraction of non-trivial knowledge from semantically annotated data streams, especially on the Web and for the Internet of Things (IoT). Often, this extraction requires expressive reasoning, which is challenging to compute on large streams. We propose Laser, a new reasoner that supports a pragmatic, non-trivial fragment of the logic LARS which extends Answer Set Programming (ASP) for streams. At its core, Laser implements a novel evaluation procedure which annotates formulae to avoid the re-computation of duplicates at multiple time points. This procedure, combined with a judicious implementation of the LARS operators, is responsible for significantly better runtimes than the ones of other state-of-the-art systems like C-SPARQL and CQELS, or an implementation of LARS which runs on the ASP solver Clingo. This enables the application of expressive logic-based reasoning to large streams and opens the door to a wider range of stream reasoning use cases.
A Family of Metrics for Clustering Algorithms
Venue: arXiv cs.DM 2017
Year: 2017

We give the motivation for scoring clustering algorithms and a metric $M : A \rightarrow \mathbb{N}$ from the set of clustering algorithms to the natural numbers which we realize as \begin{equation} M(A) = \sum_i α_i |f_i - β_i|^{w_i} \end{equation} where $α_i,β_i,w_i$ are parameters used for scoring the feature $f_i$, which is computed empirically.. We give a method by which one can score features such as stability, noise sensitivity, etc and derive the necessary parameters. We conclude by giving a sample set of scores.
Building Detection from Satellite Images on a Global Scale
Venue: arXiv cs.CV 2017
Year: 2017

In the last several years, remote sensing technology has opened up the possibility of performing large scale building detection from satellite imagery. Our work is some of the first to create population density maps from building detection on a large scale. The scale of our work on population density estimation via high resolution satellite images raises many issues, that we will address in this paper. The first was data acquisition. Labeling buildings from satellite images is a hard problem, one where we found our labelers to only be about 85% accurate at. There is a tradeoff of quantity vs. quality of labels, so we designed two separate policies for labels meant for training sets and those meant for test sets, since our requirements of the two set types are quite different. We also trained weakly supervised footprint detection models with the classification labels, and semi-supervised approaches with a small number of pixel-level labels, which are very expensive to procure.
Deep Kernels for Optimizing Locomotion Controllers
Venue: arXiv cs.RO 2017
Year: 2017

Sample efficiency is important when optimizing parameters of locomotion controllers, since hardware experiments are time consuming and expensive. Bayesian Optimization, a sample-efficient optimization framework, has recently been widely applied to address this problem, but further improvements in sample efficiency are needed for practical applicability to real-world robots and high-dimensional controllers. To address this, prior work has proposed using domain expertise for constructing custom distance metrics for locomotion. In this work we show how to learn such a distance metric automatically. We use a neural network to learn an informed distance metric from data obtained in high-fidelity simulations. We conduct experiments on two different controllers and robot architectures. First, we demonstrate improvement in sample efficiency when optimizing a 5-dimensional controller on the ATRIAS robot hardware. We then conduct simulation experiments to optimize a 16-dimensional controller for a 7-link robot model and obtain significant improvements even when optimizing in perturbed environments. This demonstrates that our approach is able to enhance sample efficiency for two different controllers, hence is a fitting candidate for further experiments on hardware in the future.
Tartan: Accelerating Fully-Connected and Convolutional Layers in Deep Learning Networks by Exploiting Numerical Precision Variability
Venue: arXiv cs.NE 2017
Year: 2017

Tartan (TRT), a hardware accelerator for inference with Deep Neural Networks (DNNs), is presented and evaluated on Convolutional Neural Networks. TRT exploits the variable per layer precision requirements of DNNs to deliver execution time that is proportional to the precision p in bits used per layer for convolutional and fully-connected layers. Prior art has demonstrated an accelerator with the same execution performance only for convolutional layers. Experiments on image classification CNNs show that on average across all networks studied, TRT outperforms a state-of-the-art bit-parallel accelerator by 1:90x without any loss in accuracy while it is 1:17x more energy efficient. TRT requires no network retraining while it enables trading off accuracy for additional improvements in execution performance and energy efficiency. For example, if a 1% relative loss in accuracy is acceptable, TRT is on average 2:04x faster and 1:25x more energy efficient than a conventional bit-parallel accelerator. A Tartan configuration that processes 2-bits at time, requires less area than the 1-bit configuration, improves efficiency to 1:24x over the bit-parallel baseline while being 73% faster for convolutional layers and 60% faster for fully-connected layers is also presented.
How Often Should CSI be Updated for Massive MIMO Systems with Massive Connectivity?
Venue: arXiv cs.IT 2017
Year: 2017

Massive multiple-input multiple-output (MIMO) systems need to support massive connectivity for the application of the Internet of things (IoT). The overhead of channel state information (CSI) acquisition becomes a bottleneck in the system performance due to the increasing number of users. An intermittent estimation scheme is proposed to ease the burden of channel estimation and maximize the sum capacity. In the scheme, we exploit the temporal correlation of MIMO channels and analyze the influence of the age of CSI on the downlink transmission rate using linear precoders. We show the CSI updating interval should follow a quasi-periodic distribution and reach a trade-off between the accuracy of CSI estimation and the overhead of CSI acquisition by optimizing the CSI updating frequency of each user. Numerical results show that the proposed intermittent scheme provides significant capacity gains over the conventional continuous estimation scheme.
Learning Pixel-Distribution Prior with Wider Convolution for Image Denoising
Venue: arXiv cs.CV 2017
Year: 2017

In this work, we explore an innovative strategy for image denoising by using convolutional neural networks (CNN) to learn pixel-distribution from noisy data. By increasing CNN's width with large reception fields and more channels in each layer, CNNs can reveal the ability to learn pixel-distribution, which is a prior existing in many different types of noise. The key to our approach is a discovery that wider CNNs tends to learn the pixel-distribution features, which provides the probability of that inference-mapping primarily relies on the priors instead of deeper CNNs with more stacked nonlinear layers. We evaluate our work: Wide inference Networks (WIN) on additive white Gaussian noise (AWGN) and demonstrate that by learning the pixel-distribution in images, WIN-based network consistently achieves significantly better performance than current state-of-the-art deep CNN-based methods in both quantitative and visual evaluations. \textit{Code and models are available at \url{https://github.com/cswin/WIN}}.