With the advent of large labelled datasets and highcapacity models, the performance of machine vision systems has been improving rapidly. However, the technology has still major limitations, starting from the fact that different vision problems are still solved by different models, trained from scratch or fine-tuned on the target data. The human visual system, in stark contrast, learns a universal representation for vision in the early life of an individual. This representation works well for an enormous variety of vision problems, with little or no change, with the major advantage of requiring little training data to solve any of them. In this paper we investigate whether neural networks may work as universal representations by studying their capacity in relation to the "size" of a large combination of vision problems. We do so by showing that a single neural network can learn simultaneously several very different visual domains (from sketches to planktons and MNIST digits) as well as, or better than, a number of specialized networks. However, we also show that this requires to carefully normalize the information in the network, by using domainspecific scaling factors or, more generically, by using an instance normalization layer.
The term "universal representation" was coined by REF and refers to an image representation that works equally well for a large number of visual domains.
Universal representations:The missing link between faces, text, planktons, and cat breeds
ArXiv
ArXiv
Computer Science Mathematics