tokens: In order to analyze this problem for synchronous Stochastic Gradient Descent ( SGD ) training of deep learning models , we developed a performance model of computation time and communication latency under three different system architectures : Parameter Server ( PS ) , peer - to - peer ( P2P ) , and Ring allreduce ( RA ) .