Immediately following the war, he returned to Palestine. After the establishment of the State of Israel, he fought in the 1948 Arab–Israeli War, serving as an officer in the battles for Latrun. His intelligence experience during World War II was seen as a valuable asset, and he subsequently became head of the IDF Military Intelligence Branch, a position in which he served from 1948 to 1950 and again from 1959 to 1962. From 1950 to 1954, he served as defence attaché at the Israeli Embassy in the United States. Herzog left Washington in September 1954. A State Department official had informed him that he was about to be declared "persona non grata". The decision to expel him had been taken following an FBI investigation into his attempt to recruit a Jordanian diplomat. He retired from the IDF in 1962 with the rank of major-general.