page_title: Autoencoder
section_title: Advantages of Depth
hierarchical_section_title: Autoencoder / Advantages of Depth
caption_attribution_description: English: Schematic picture of an autoencoder architecture
context_page_description: An autoencoder is a type of artificial neural network used to learn efficient data codings in an unsupervised manner. The aim of an autoencoder is to learn a representation for a set of data, typically for dimensionality reduction, by training the network to ignore signal “noise”. Along with the reduction side, a reconstructing side is learnt, where the autoencoder tries to generate from the reduced encoding a representation as close as possible to its original input, hence its name. Several variants exist to the basic model, with the aim of forcing the learned representations of the input to assume useful properties. Examples are the regularized autoencoders, proven effective in learning representations for subsequent classification tasks, and Variational autoencoders, with their recent applications as generative models. Autoencoders are effectively used for solving many applied problems, from face recognition to acquiring the semantic meaning of words.
context_section_description: Autoencoders are often trained with only a single layer encoder and a single layer decoder, but using deep encoders and decoders offers many advantages.
  Depth can exponentially reduce the computational cost of representing some functions.
  Depth can exponentially decrease the amount of training data needed to learn some functions.
  Experimentally, deep autoencoders yield better compression compared to shallow or linear autoencoders.
caption: Schematic structure of an autoencoder with 3 fully connected hidden layers. The code (z, or h for reference in the text) is the most internal layer.
exif: {}