Before revision:  We find that word error rate can be reduced by a large margin by pre-training the acoustic  representation with AWEs, and additional (smaller) gains can be obtained by pre-training the word prediction layer with AGWEs.
After revision:  We find that word error rate can be reduced by a large margin by pre-training the acoustic segment representation with AWEs, and additional (smaller) gains can be obtained by pre-training the word prediction layer with AGWEs.