Church officials say he never ruled out a visit to North Korea .