Make sure only training data is applied on corresponding trained weights