instruction:
In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
question:
The EM-algorithm is a general procedure to get maximum likelihood estimates if part of the observations on the variables of a network are missing. In this paper a stochastic version of the algorithm is adapted to probabilistic neural networks describing the associative dependency of variables. These networks have a proba­ bility distribution, which is a special case of the distribution generated by proba­ bilistic inference networks. Hence both types of networks can be combined al­ lowing to integrate probabilistic rules as well as unspecified associations in a sound way. The resulting network may have a number of interesting features including cycles of probabilistic rules, hidden 'un­ observable' variables, and uncertain and contradictory evidence.
answer:
Integrating Probabilistic Rules into Neural Networks: A Stochastic EM Learning Algorithm


question:
Machine learning is increasingly used in security-critical applications, such as autonomous driving, face recognition and malware detection. Most learning methods, however, have not been designed with security in mind and thus are vulnerable to different types of attacks. This problem has motivated the research field of adversarial machine learning that is concerned with attacking and defending learning methods. Concurrently, a different line of research has tackled a very similar problem: In digital watermarking information are embedded in a signal in the presence of an adversary. As a consequence, this research field has also extensively studied techniques for attacking and defending watermarking methods. The two research communities have worked in parallel so far, unnoticeably developing similar attack and defense strategies. This paper is a first effort to bring these communities together. To this end, we present a unified notation of black-box attacks against machine learning and watermarking that reveals the similarity of both settings. To demonstrate the efficacy of this unified view, we apply concepts from watermarking to machine learning and vice versa. We show that countermeasures from watermarking can mitigate recent model-extraction attacks and, similarly, that techniques for hardening machine learning can fend off oracle attacks against watermarks. Our work provides a conceptual link between two research fields and thereby opens novel directions for improving the security of both, machine learning and digital watermarking.
answer:
Fraternal Twins: Unifying Attacks on Machine Learning and Digital Watermarking


question:
Neuroimage analysis usually involves learning thousands or even millions of variables using only a limited number of samples. In this regard, sparse models, e.g. the lasso, are applied to select the optimal features and achieve high diagnosis accuracy. The lasso, however, usually results in independent unstable features. Stability, a manifest of reproducibility of statistical results subject to reasonable perturbations to data and the model (Yu 2013), is an important focus in statistics, especially in the analysis of high dimensional data. In this paper, we explore a nonnegative generalized fused lasso model for stable feature selection in the diagnosis of Alzheimer’s disease. In addition to sparsity, our model incorporates two important pathological priors: the spatial cohesion of lesion voxels and the positive correlation between the features and the disease labels. To optimize the model, we propose an efficient algorithm by proving a novel link between total variation and fast network flow algorithms via conic duality. Experiments show that the proposed nonnegative model performs much better in exploring the intrinsic structure of data via selecting stable features compared with other state-of-the-arts. Introduction Neuroimage analysis is challenging due to its high feature dimensionality and data scarcity. Sparse models such as the lasso (Tibshirani 1996) have gained great reputation in statistics and machine learning, and they have been applied to the analysis of such high dimensional data by exploiting the sparsity property in the absence of abundant data. As a major result, automatic selection of relevant variables/features by such sparse formulation achieves promising performance. For example, in (Liu, Zhang, and Shen 2012), the lasso model was applied to the diagnosis of Alzheimer’s disease (AD) and showed better performance than the support vector machine (SVM), which is one of the state-of-the-arts in brain image classification. However, in statistics, it is known that the lasso does not always provide interpretable results because of its instability (Yu 2013). “Stability” here means the reproducibility of statistical results subject to reasonable perturbations to data and Copyright c © 2015, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. the model. (These perturbations include the often used Jacknife, bootstrap and cross-validation.) This unstable behavior of the lasso model is critical in high dimensional data analysis. The resulting irreproducibility of the feature selection are especially undesirable in neuroimage analysis/diagnosis. However, unlike the problems such as registration and classification, the stability issue of feature selection is much less studied in this field. In this paper we propose a model to induce more stable feature selection from high dimensional brain structural Magnetic Resonance Imaging (sMRI) images. Besides sparsity, the proposed model harnesses two important additional pathological priors in brain sMRI: (i) the spatial cohesion of lesion voxels (via inducing fusion terms) and (ii) the positive correlation between the features and the disease labels. The correlation prior is based on the observation that in many brain image analysis problems (such as AD, frontotemporal dementia, corticobasal degeneration, etc), there exist strong correlations between the features and the labels. For example, gray matter of AD is degenerated/atrophied. Therefore, the gray matter values (indicating the volume) are positively correlated with the cognitive scores or disease labels {-1,1}. That is, the less gray matter, the lower the cognitive score. Accordingly, we propose nonnegative constraints on the variables to enforce the prior and name the model as “non-negative Generalized Fused Lasso” (nGFL). It extends the popular generalized fused lasso and enables it to explore the intrinsic structure of data via selecting stable features. To measure feature stability, we introduce the “Estimation Stability” recently proposed in (Yu 2013) and the (multi-set) Dice coefficient (Dice 1945). Experiments demonstrate that compared with existing models, our model selects much more stable (and pathological-prior consistent) voxels. It is worth mentioning that the non-negativeness per se is a very important prior of many practical problems, e.g. (Lee and Seung 1999). Although nGFL is proposed to solve the diagnosis of AD in this work, the model can be applied to more general problems. Incorporating these priors makes the problem novel w.r.t the lasso or generalized fused lasso from an optimization standpoint. Although off-the-shelf convex solvers such as CVX (Grant and Boyd 2013) can be applied to solve the optimization, it hardly scales to high-dimensional problems in feasible time. In this regard, we propose an efficient algoar X iv :1 50 3. 07 50 8v 1 [ cs .L G ] 2 5 M ar 2 01 5 rithm that solves the nGFL problem exactly. We generalize the proximal gradient methods (such as FISTA) (Beck and Teboulle 2009) to solve our constrained optimization and prove its convergence. We then show that by using an element-wise post-processing, the resulting proximal operator can be reduced to the total variation (TV) problem. It is known that TV can be solved by parametric flow algorithms (Chambolle and Darbon 2009; Xin et al. 2014). In the present study, we provide a novel equivalence via conic duality, which gives us a minimum quadratic cost flow formulation (Hochbaum and Hong 1995). Fast flow algorithms (including parametric flow) are then easily applied. In practice, our algorithm runs hundreds of times faster than CVX at the same precision and can scale to high-dimensional problems. Related work. In addition to sparsity, people leverage underlying data structures and introduce stronger priors such as the structured sparsity (Jacob, Obozinski, and Vert 2009) to increase model stability. However, for voxel-based sMRI data analysis, handcrafted grouping of the voxels or sub-structures may not coincide with various pathological topology priors. Consequently, group lasso (with overlap) (Jacob, Obozinski, and Vert 2009; Jenatton et al. 2012; Rao et al. 2013) is not an ideal model to the problem. In contrast, the graph-based structured sparse models adapt better to such a situation. The most popular one is referred here as LapL1, which adopts l2 norm regularization of neighborhood variable difference (e.g. (Ng and Abugharbieh 2011; Grosenick et al. 2013)). However, as we will show in the experiments, these models select many more features than necessary. Very recently, generalized fused lasso or total variation has been successful applied to brain image analysis problems inducing the l1 difference (Gramfort, Thirion, and Varoquaux 2013; Xin et al. 2014). In the experiments, we show that by including an extra nonnegative constraint, the features selected by our model is much more stable than that of such unconstrained models. A very recent work (Avants et al. 2014) also explored this positive correlation (partially supporting our assumption), but the problem formulation was quite different: neither structural assumption was considered, nor the stability of feature selection was discussed. From the optimization standpoint, the applied framework is similar to that of (Xin et al. 2014) but two key differences exist: (1) the FISTA and soft-thresholding process applied in (Xin et al. 2014) do not generalize to constrained optimization problems, we show important modifications and provide theoretical proof; (2) we propose a novel understanding of TV’s relation with flow problems via conic duality and prove that the minimum norm point problem of (Xin et al. 2014) is a special case of our framework. The Proposed Method Nonnegative Generalized Fused Lasso (n2GFL) Let {(xi, yi)}i=1 be a set of samples, where xi ∈ R and yi ∈ R are features and labels, respectively. Also, we denote Although different names are given in e.g. (Ng and Abugharbieh 2011; Grosenick et al. 2013), they are in fact fundamentally applying the graph Laplacian smoothing. by X ∈ Rd×N and y ∈ R the concatenations of xi and yi. Then, we consider the formulation min β∈Rd l(β;X,y) + λ1 d ∑
answer:

Stable Feature Selection from Brain sMRI