The Union of South Africa objected to South-West Africa coming under UN control and refused to allow the territory 's transition to independence , regarding it as a fifth province ( even though it was never formally incorporated into South Africa ) .
However , South Africa refused , regarding the territory as its fifth province ( even though it was never formally incorporated into the Union of South Africa ) .