Random Partition Relaxation for Training Binary and Ternary Weight Neural Network
We present Random Partition Relaxation, a method for strong quantization of the parameters of convolutional neural networks to binary and ternary values.Starting from a pretrained model, we first quantize the weights and then relax random partitions of them to their continuous values for retraining before quantizing them again and switching to another weight partition for further adaptation. We empirically evaluate the performance of RPR with ResNet-18, ResNet-50 and GoogLeNet on the ImageNet classification task for binary and ternary weight networks.We show accuracies beyond the state-of-the-art for binary- and ternary-weight GoogLeNet and competitive performance for ResNet-18 and ResNet-50 using a SGD-based training method that can easily be integrated into existing frameworks.
State-of-the-art training method for binary and ternary weight networks based on alternating optimization of randomly relaxed weight partitionsThe paper proposes a new training scheme of optimizing a ternary neural network.Authors propose RPR, a way to randomly partition and quantize weights and train the remaining parameters followed by relaxation in alternate cycles to train quantized models.