Stawinoga was born on Krążkowy, near Kępno, Poland. Little is known about his early life, but he was thought to have been involved in the 1939 Soviet invasion of Poland, before emigrating to the UK in the 1940s. Correspondence with his nephew Roland Stawinoga, the grandson of Józef's father, Robert Stawinoga, however, revealed that he served for the German armed forces in Africa under General Rommel; after the defeat at El-Alamein in November 1942, he became an English prisoner of war in Egypt. Roland Stawinoga said, "Someone noticed that he spoke good Polish and that he had a Polish mother, so he was allowed to join the Polish army in England. That was how he survived, but he had suffered mental problems because of his experiences in the war."