The first was on the Fifth Avenue site of the Empire State Building .
The first , designed by architect Henry J. Hardenbergh , was on the Fifth Avenue site of the Empire State Building .