Yes, Google translate uses a hybrid model of transformers and RNNs to get good performance.