During the 1936-1939 Arab revolt in Palestine, Yermiya joined the Special Night Squads. In 1938, he was among the founders of kibbutz Eilon, and became a member of the kibbutz. He participated in a battle for kibbutz Hanita, and was appointed regional commander of the area. After World War II, broke out, Yermiya joined the British Army, and served in Palestinian Transport Corps. He participated in the North African and Middle Eastern fronts, and the invasions of Italy and Germany. At the end of the war, he was a member of the Haganah's elite strike unit, the Palmach, and participated in Palmach operations to smuggle illegal Jewish immigrants into Palestine as part of Aliyah Bet.