Original: He returned to Japan only once, to marry his wife.
Simplified: He only went back to Japan once, and that was to marry his wife.