In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.

[EX Q]: Observable operator models (OOMs) and related models are one of the most important and powerful tools for modeling and analyzing stochastic systems. They can exactly describe dynamics of finite-rank systems, and be efficiently learned from data by moment based algorithms. Almost all OOM learning algorithms are developed based on the assumption of equilibrium data which is very difficult to guarantee in real life, especially for complex processes with large time scales. In this paper, we derive a nonequilibrium learning algorithm for OOMs, which dismisses this assumption and can effectively extract the equilibrium dynamics of a system from nonequilibrium observation data. In addition, we propose binless OOMs for the application of nonequilibrium learning to continuous-valued systems. In comparison with the other OOMs with continuous observations, binless OOMs can achieve consistent estimation from nonequilibrium data with only linear computational complexity.
[EX A]: Spectral learning of dynamic systems from nonequilibrium data∗

[EX Q]: Formal languages for probabilistic modeling enable re-use, modularity, and descriptive clarity, and can foster generic inference techniques. We introduce Church, a universal language for describing stochastic generative processes. Church is based on the Lisp model of lambda calculus, containing a pure Lisp as its deterministic subset. The semantics of Church is defined in terms of evaluation histories and conditional distributions on such histories. Church also includes a novel language construct, the stochastic memoizer, which enables simple description of many complex non-parametric models. We illustrate language features through several examples, including: a generalized Bayes net in which parameters cluster over trials, infinite PCFGs, planning by inference, and various non-parametric clustering models. Finally, we show how to implement query on any Church program, exactly and approximately, using Monte Carlo techniques.
[EX A]: Church: a language for generative models

[EX Q]: Stacked denoising auto encoders (DAEs) are well known to learn useful deep representations, which can be used to improve supervised training by initializing a deep network. We investigate a training scheme of a deep DAE, where DAE layers are gradually added and keep adapting as additional layers are added. We show that in the regime of mid-sized datasets, this gradual training provides a small but consistent improvement over stacked training in both reconstruction quality and classification error over stacked training on MNIST and CIFAR datasets. 1 GRADUAL TRAINING OF DENOISING AUTOENCODERS We test here gradual training of deep denoising auto encoders, training the network layer-by-layer, but lower layers keep adapting throughout training. To allow lower layers to adapt continuously, noise is injected at the input level. This training procedure differs from stack-training of auto encoders (Vincent et al., 2010) More specifically, in gradual training, the first layer of the deep DAE is trained as in stacked training, producing a layer of weights w1. Then, when adding the second layer autoencoder, its weights w2 are tuned jointly with the already-trained weights w1. Given a training sample x, we generate a noisy version x̃, feed it to the 2-layered DAE, and compute the activation at the subsequent layers h1 = Sigmoid(w > 1 x), h2 = Sigmoid(w > 2 h1) and y = Sigmoid(w ′> 3 h2). Importantly, the loss function is computed over the input x, and is used to update all the weights including w1. Similarly, if a 3rd layer is trained, it involves tuning w1 and w2 in addition to w3 and w′ 4. 2 EXPERIMENTAL PROCEDURES We compare the performance of gradual and stacked training in two learning setups: an unsupervised denoising task, and a supervised classification task initialized using the weights learned in an unsupervised way. Evaluations were made on three benchmarks: MNIST, CIFAR-10 and CIFAR100, but only show here MNIST results due to space constraints. We used a test subset of 10,000 samples and several sizes of training-set all maintaining the uniform distribution over classes. Hyper parameters were selected using a second level of cross validation, including the learning rate, SGD batch size, momentum and weight decay. In the supervised experiments, training was ’early stopped’ after 35 epochs without improvement. The results reported below are averages over 3 train-validation splits. Since gradual training involves updating lower layers, every presentation of a sample involves more weight updates than in a single-layered DAE. To compare stacked and gradual training on a common ground, we limited gradual training to use the same budget of weight update steps as stacked training. For example, when training the second layer for n epochs in gradual training, we allocate 2n training epochs for stacked training (details in the full paper). 1 ar X iv :1 50 4. 02 90 2v 1 [ cs .L G ] 1 1 A pr 2 01 5 Accepted as a workshop contribution at ICLR 2015 a) Unsupervised Training b) Supervised training 0 0.25 0.5 10.4 10.5 10.6 10.7
[EX A]:
GRADUAL TRAINING METHOD FOR DENOISING AUTO ENCODERS