Convolutional networks to relate images Zusammenfassung: Within recent years convolutional neural networks have experienced a come back and are currently dominating the field of computer vision.In contrast to previously used methods they allow for full data based end to end training without the need to manually engineer features.On the downside huge amounts of annotated data are needed to train a network successfully.Furthermore the architectural design of these networks and other hyperparameters are sometimes hard to choose especially when predicting high resolution outputs from images.In this thesis we show how networks can be trained without manually labeled data by using surrogate tasks strong augmentation and synthetically generated training data.We demonstrate how generic image features for classification and matching can be learned unsupervisedly by augmenting random images to form surrogate classes.We also train networks to estimate the canonical orientation of rotated images by augmenting a large set of natural unlabeled images.In the main part of this thesis we present a new network architecture that can be used for any pixel level regression task.We prove its value by creating the first end to end learning approach to compute optical flow and extend it to disparity and scene flow.All of this is done by using synthetically generated training data which we rendered using a custom version of the Blender rendering suite