System: Instructions: Compose a comprehensive reply to the query using the search results given. Cite each reference using [Page Number] notation (every result has this number at the beginning).Citation should be done at the end of each sentence. If the search results mention multiple subjects with the same name, create separate answers for each. Only include information found in the results and don't add any additional information. Make sure the answer is correct and don't output false content. Ignore outlier search results which has nothing to do with the question. You can give additional information using you own knowledge but you have to start by the tag [Own knowledge]. Only answer what is asked. The answer should be short and concise. Answer step-by-step.

User: search results:

[Page no. 3] "Θ2)||2 (1) where ||·|| is the L2 norm, Θ1 and Θ2 are the parameters of the encoder (resp. decoder) part of the autoencoder, and AE is the function implemented by the autoencoder. The goal is to train the model (AE(X, Θ1, Θ2)) with the aim of reconstructing the input X as closely as possible. In a typical autoencoder structure, the number of nodes in the output layer is the same as in the input layer and the net- work structure is layered and symmetric. In the encoder part, the number of neurons in the internal layers decrease gradually. Symmetrically, it increases in the decoder part. Therefore, the only way to reconstruct the original data accurately is to learn 2018 International Joint Conference on Neural Networks (IJCNN)  Θ1, Θ2 so that the encoding-decoding process achieves good data compression and reconstruction abilities. Bottleneck Layer Encoder Decoder Approximate Reconstruction (a) Bottleneck Layer Encoder Decoder Prediction Reconstruction (b) Fig. 1. Layered architecture of an autoencoder (a) and of a semi-supervised autoencoder (b). C. Semi-supervised autoencoders A semi-supervised autoencoder (SSAE) [11], [19] is a particular kind of neural network architecture that solves two different tasks at the same time: i) a data reconstruction task via a classic encoding-decoding schema and ii) a classiﬁcation task through the encoding part of the network. In this work, we employ a semi-supervised autoencoder in which the bottleneck layer of the autoencoder also deals with the prediction task. Figure 1(b) visually presents our semi-supervised autoencoder. In addition to the reconstruction task (Equation 1), a part of the parameters is also optimized to address the classiﬁcation task. In particular, the loss function for the classiﬁcation task is the categorical cross-entropy between the labels and the prediction of the SSAE: Lcl(Θ1, Θ3) = − 1 |X| |X| � j=1 |C| �"

[Page no. 3] "c=1 yjc · log(ˆyjc) (2) where yj. is the one hot encoding of the class label for the example j, ˆyj. = SSAE(Xj, Θ1, Θ3) is the probability distribution of the SSAE prediction over the set of possible labels, Θ1 are the parameters of the encoder (the same as in Equation 1), and Θ3 are the parameters used to perform classi- ﬁcation starting from the bottleneck layer of the autoencoder. The overall loss function optimized by the semi-supervised autoencoder is then: LSSAE(Θ1, Θ2, Θ3) = Lae + λLcl (3) where Lae = 1 |X| |X| � i=1 ||Xi − SSAE(Xi, Θ1, Θ2)||2 (4) Lcl = − 1 |Xl| |Xl| � j=1 |C| � c=1 yjc · log(ˆyjc) (5) In our architecture, the two loss functions involve two sets of data: Lae is trained on the whole dataset X, while Algorithm 1 Semi-supervised autoencoder optimization Require: Xu, Xl, N EPOCHS, fl size, b size Ensure: Θ1, Θ2, Θ3. 1: i = 0 2: X = Xu ∪ {x|(x, y) ∈ Xl} 3: initSSAE(fl size, b size) 4: while i < N EPOCHS do 5: Update Θ1 and Θ2 by descending the gradient: 6: ∇Θ1,Θ2 1 |X| �|X| i=1 ||Xi − SSAE(Xi, Θ1, Θ2)||2 7: Update Θ1, Θ2 and Θ3 by descending the gradient: 8: ∇Θ1,Θ2,Θ3 1 |Xl| �|Xl| j=1 ||Xj − SSAE(Xj, Θ1, Θ2)||2 − λ � 1 |Xl| �|Xl| j=1 yj · log(SSAE(Xj, Θ1, Θ3)) � 9: i = i + 1 10: end while 11: return Θ1, Θ2, Θ3 Lcl is learnt by exploiting only the labeled subset Xl. This supports the learning of low-dimensional representations that well summarize the information carried by the original data while taking into account the background knowledge supplied by the few available labeled examples. In the following, we provide the details of the"

[Page no. 4] "procedure we employ for the optimization of Equation 3. D. Semi-Supervised autoencoder structure and optimization The internal structure of our network uses Rectiﬁer Linear Units (ReLU [17]) as activation functions for all the encoder and decoder layers with the exception of the last layer. In the latter, in fact, we employ a sigmoid activation function. To this purpose, all data attributes are normalized in the range [0, 1] before feeding the network. As regards the classiﬁcation task, we link the bottleneck layer of the autoencoder to the classiﬁcation output by a linear activation function followed by a softmax. The optimization strategy we adopt is reported in Al- gorithm 1. During each epoch, the algorithm performs the minimization of the reconstruction loss involving Θ1 and Θ2 parameters on the whole dataset (line 5-6) and the minimiza- tion of the reconstruction and classiﬁcaton loss involving Θ1, Θ2 and Θ3 parameters for the subset of labeled instances (line 7-8). The optimization is realized via the use of mini-batches. IV. THE MSAEClust MODEL In this section, we present the details of our Multiresolu- tion Semi-supervised AutoEncoder-based Clustering, namely MSAEClust. It follows a multiresolution schema sketched in Algorithm 2. The intuition behind MSAEClust is related to the low-dimensional representation supplied by a generic semi-supervised autoencoder. As we discussed before, a semi- supervised autoencoder produces a low-dimensional embed- ding of the original data addressing a reconstruction task (on the whole dataset) and a classiﬁcation task (on the set of labeled examples) at the same time. Following the general idea of ensemble learning [12] in which a committee is preferred to one single model, we leverage an ensemble of semi- supervised autoencoders with the aim of computing different low-dimensional embeddings. However, directly combining a 2018 International Joint Conference on Neural Networks (IJCNN)  Algorithm 2 MultiResolution Strategy"

[Page no. 2] "by leveraging an adap- tive learning strategy to train an ensemble of stacked semi- supervised autoencoder architectures. III. SEMI-SUPERVISED CLUSTERING WITH AUTOENCODERS In this section, we provide the theoretical foundations of our semi-supervised clustering approach. Before presenting the core part of our method, we introduce some notations and preliminaries. A. Semi-supervised clustering In a typical semi-supervised learning scenario there are two different sets of examples: a set Xu = {xi}N i=1 of N instances with no available class information, and a set Xl = {(xj, yj)}M j=1 of M instances for which the class information is available. In particular, each data instance xj ∈ Xl is associated to a class variable yj ∈ C, where C the set of possible labels. The general assumption is that |Xu| >> |Xl| with the extreme (and more realistic) case where only very few examples (e.g., one) per class exist. The goal of semi-supervised clustering is to group together data examples belonging to X = Xu ∪ Xl in k clusters exploiting as much as possible the knowledge provided by the labeled set Xl. B. Autoencoders Figure 1(a) visually summarizes the structure of a generic autoencoder. An autoencoder is a particular kind of feed- forward multi-layer neural networks that performs successive linear (or non linear) transformations with the aim of re- constructing the original data. An autoencoder is composed of two parts: i) an encoder that compresses the original data into a low-dimensional representation, and ii) a decoder that reconstructs the original data from the low-dimensional representation. The outputs of the smallest layer constitute the low-dimensional representation of the original data. This layer is usually named bottleneck layer (the central layer in Figure 1(a)). Formally, the autoencoder optimizes the following Loss funciton: Lae(Θ1, Θ2) = 1 |X| |X| � i=1 ||Xi − AE(Xi, Θ1,"

[Page no. 5] "the behavior and the performances of our algorithm. First, we compare MSAEClust to other semi- supervised clustering methods. Then, we perform an in-depth analysis of MSAEClust’s results to assess its sensitivity w.r.t. to the ensemble size. Finally, we investigate qualitatively the new embedding generated by our algorithm through visual inspection. A. Competitors For the comparative study, we consider the following com- petitors: Information Theoretic metric learning (ITML), pair- wise constrained k-means (PCKmeans), metric-pairwise con- strained k-means (MPCKmeans), and seeded k-means (Seed- edKmeans). ITML exploits information theory to learn a suit- able metric space that meets the supervision supplied as side- information on the distance function [9]. Since it only supplies a new metric space, we use k-means to successively obtain the data partition. PCKmeans leverages must-link and cannot- link constraints to modify the internal objective function [3]. MPCKmeans combines metric-learning and constrained clus- tering to exploit as much as possible the supplied supervi- sion [5]. SeededKmeans uses labeled instances to initialize a standard k-means clustering algorithm [2]. Finally, we also include k-means in the comparative study, so as to have a simple fully unsupervised baseline. B. Datasets To evaluate the comparative and in-depth analysis results, the experiments are performed over six publicly available datasets published in the UCI machine learning repository1. TABLE I MAIN CHARACTERISTICS OF THE DATASETS Dataset # Instances # Features # Classes Splice 3190 60 3 Landsat 4 435 36 6 Mushroom 8 124 22 2 Spambase 4 601 57 2 Letter 20 000 16 26 USPS 9 298 256 10 1http://archive.ics.uci.edu/ml/index.php 2018 International Joint Conference on Neural Networks (IJCNN)  (a) Splice (b) Landsat (c) Mushroom (d) Spambase (e) Letter (f) USPS Fig. 2. NMI values of the comparative study (best viewed in color version, available in electronic format). A summary of the information about the"

Query: Why this auto encoder is Multiresolution ?
Answer:
Assistant: