Rall began his service with the Imperial Navy, undertaking training and instruction at the Kronstadt Naval Engineering School and the Naval Cadet Corps, with practical experience as a midshipman in the Black Sea Fleet. He served in the Baltic Fleet during the First World War, graduating to his own commands towards the end of the war. A popular officer for his attention to the lower ranks, he supported the Bolsheviks after the Russian Revolution, and was elected to one of the ship committees. During the Russian Civil War he fought against the White Finns and British interventionist forces. As an experienced officer he was tasked with several staff positions after the war, and took a leading role in re-establishing the fleet's strength in the early Soviet period. He took a particular interest in training new officers, and took several roles in the naval academies.