At the start of the Russo-Japanese War (1904–1905), Zarubaev was made commander of the Fourth Siberian Army Corps, which was composed of the Second and Third Siberian Reserve Brigades. He successfully repulsed attacks by Japanese general Yasukata Oku during the Battle of Tashihchiao, but was ordered to retreat by General Aleksey Kuropatkin rather than to follow up on his success. Likewise, at the Battle of Liaoyang, he successfully held his position against repeated Japanese attacks, but was again refused reinforcements by Kuropatkin and ordered to retreat. At the Battle of Shaho, his forces were largely held in reserve, and were used to reinforce the line covering Kuropatkin's retreat. During the Battle of Mukden, Zarubaev's forces held their positions for ten days from 12–22 February. From 23 February, Zarubaev commanded a combined force of both the Fourth Siberian Army Corps and the First Siberian Army Corps. However, this combined force was cancelled on 25 February, and after Kuropatkin was replaced by General Nikolai Linevich, Zarubaev was made temporary commander of the Russian First Manchurian Army.