During World War II, his father, a colonel, fought under General Douglas MacArthur during the Japanese invasion of the Philippines. Two weeks into the conflict, newspaper reports described Colonel Hilsman as still holding Davao on the island of Mindanao. Later reports reflected his retreat to Malaybalay after he had faced overwhelming Japanese forces, followed by another move onto the island of Negros and he was captured by the Japanese once all the islands had surrendered in 1942.