After serving in the prison for less than a year, the Second Sino-Japanese War broke out, his senior classmate Wang Yaowu successfully bailed him out of prison and appointed him as a regiment commander, Zhang participated the brutal street fighting in the Battle of Shanghai and Battle of Nanjing. In 1938 he was promoted to brigade commander, during the Battle of Wanjialing of the Wuhan theatre, he personally led a detachment that cut off the Japanese retreat route and ensured the final Chinese victory. In 1939 he was wounded in his left leg during the Battle of Nanchang. Although his doctor told after one more month of treatment his wounds could be healed, he returned to the army and participated in the Battle of Changsha. As a result, he was unable to recover from his injuries and his colleagues nicknamed him The lame legged general. Afterward, he was promoted to assistant division commander of the 51st division and deputy commander of 74th corps. In 1943, Zhang personally led a task force which relieved the besieged Chinese defenders in the ferocious Battle of Changde and forced the Imperial Japanese Army to retreat, and Chiang Kai-shek called him the model soldier. Although he failed to save General Fang Xianjue from the Japanese attacks during the Defense of Hengyang in 1944, his unit fought well and he tried his best, so the National War Council did not take any action against him. After this rather unhappy episode in his military career, his old superior officer Wang Yaowu helped him again by sending Zhang to the Army War College to study, and Zhang graduated with top grades. In the May 1945, he fought under Wang Yaowu in the Battle of West Hunan and was decorated with Order of Precious Tripod and Medal of Freedom, and He Yingqin promoted him as commander of 74th corps.