# Quotient Network-A Network Similar to ResNet but Learning Quotients
## 1 Introduction  
Convolutional neural networks have demonstrated strong performance in computer vision tasks [1; 2; 3; 4]. In their continuous performance improvements, depth is the key factor in success [5; 6]. In order to successfully train deeper networks, in addition to initialization regularization methods [7; 8; 9], a landmark breakthrough that cannot be ignored is the ResNet method [10]. It changes the learning target to the residual value through a shortcut so that the network does not need to learn new features from scratch but learns the difference between the new and old features, thereby reducing the learning difficulty. When the weight parameters are relatively small, it is easier to maintain the identity mapping so that a deep network is at least no worse than a shallow network. However, this also brings about two problems. (1) The arithmetic difference between the new and old features is an absolute difference that does not fully utilize the size information of the old features. Obviously, for values 0.1 and 1, the effect of adding the same number 0.1 is quite different. Ignoring the size information of the old features will cause the learned intermediate values to be more sensitive to the size of the old features. The result is that the transformation is too strong for some old features and too weak for others. (2) More importantly, the CNN differs from the RNN because its input and output feature types differ. Its different layers will learn low/medium/high-level features [11]. Therefore, the difference between the new and old features is not an intermediate feature with a very clear and independent meaning. That may make the functions to be learned by the network too complex and increase the learning difficulty. These two problems will eventually cause ResNet to be unable to utilize the performance of the deep network fully. For these reasons, we propose a more natural network to make learning easier. The network changes the learning goal to the quotient between the new and the old features, so the final feature is obtained by multiplying the old features by the quotient, so we call our network the quotient network. This network can perfectly solve these two problems of ResNet. It learns the relative difference (i.e., the quotient) between the new and old features, which better allows the network to take full advantage of the size information of the old features compared with ResNet, generating quantities insensitive to the size of the old features. That makes the quotient network more influential in transforming each old feature than ResNet. Looking back at the nature in which we live, we can see that the quotient of two different classes of features is more likely to be a third feature with a meaning that is more independent and clear than the difference. For example, the value of mass divided by volume is more intuitively clear than the value of mass minus volume, with the former being density; the value of force divided by mass is more meaningful than the value of force minus mass, with the former being acceleration; the value of voltage divided by current is more straightforward than the value of voltage minus current, with the former being resistance, and so on. In the quotient network, our learning goal is the quotient of different features, which tends to have a specific meaning. Introducing such prior knowledge can reduce the complexity of the function that is to be learned by the network. Moreover, our network also has ResNet's advantages. Instead of learning the target features from scratch, it learns them by building on top of the old features. By making the activation function of the last layer of the quotient module pass through the (0, 1) point, the quotient network can also maintain the identity mapping more easily. Therefore, we have reasons to believe that our network performance is better than ResNet. We propose some empirical guidelines for designing such networks, including finding better activation functions and placement of activation functions.