factored out common preprocessing code from prepareNeuralLM and testNeuralLM