Hovda was born in Hjelmeland, the son of the teacher Johannes Hovda (1874–?) and Eli née Kleppa (1883–1939). He took his "examen artium" in 1929 and became a conscripted officer in the military in 1930, attaining the rank of major. During the German attack on Norway in 1940, he participated as an unpaid lieutenant and adjutant in the first battalion of the Rogaland Infantry Regiment No. 8 (IR8).