During the first months after Operation Barbarossa, the 22 June 1941 German invasion of the Soviet Union, Grechkin was tasked with organizing the construction of the Mius defensive line, which protected Rostov-on-Don. From 3 August to 4 September, he was the acting commander of the North Caucasus Military District. Grechkin became commander of the 56th Army operational group in October, leading it in the defense of Rostov, which ended in a retreat from the city. In conjunction with the 9th Army and the rest of the 56th Army, the operational group was able to recapture the city during the subsequent Rostov Offensive. He became the deputy commander of the 24th Army in June 1942 and took command of the 318th Rifle Division in August 1942.