In 1942, Musugiri enlisted in the King's African Rifles (KAR), beginning as a private. He later served with the KAR in Madagascar. By 1947 he was a sergeant and acted as an instructor at Kahawa Barracks in Nairobi, Kenya. While there he met future Ugandan dictator Idi Amin, who was a pupil of his. In 1957, the British administration introduced the rank of "effendi" into the KAR, which was awarded to high performing African non-commissioned officers and warrant officers (it was not a true officer classification). Musuguri was given the rank. In December 1961, Tanganyika became a sovereign state and several units of the KAR was transferred to the newly formed Tanganyika Rifles. The rank of "effendi" was shortly thereafter abandoned, and, by 1962, Musuguri had been promoted to lieutenant. During the Tanganyika Rifles mutiny of January 1964, Musuguri was stationed in Tabora. Rebellious troops, attempting to remove and replace their British officers, declared him a major.