A high-strength hot-dip galvanized steel sheet having excellent delayed fracture resistance, comprising: a base steel sheet made of steel containing: in mass%, C: 0.075 to 0.400%; Si: 0.01 to 2.00%; Mn: 0.80 to 3.50%; P: 0.0001 to 0.100%; S: 0.0001 to 0.0100%; Al: 0.001 to 2.00%; O: 0.0001 to 0.0100%; N: 0.0001 to 0.0100%; and optionally, in mass%, one kind or two or more kinds selected from Cr: 0.01 to 2.00%, Ni: 0.01 to 2.00%, Cu: 0.01 to 2.00%, Mo: 0.01 to 2.00%, B: 0.0001 to 0.0100%, W: 0.01 to 2.00%, Ti: 0.001 to 0.150%, Nb: 0.001 to 0.100%, V: 0.001 to 0.300%, 0.0001 to 0.0100 mass% in total of one kind or two or more kinds selected from Ca, Ce, Mg, Zr, La, and REM, and a balance being composed of Fe and inevitable impurities; and a hot-dip galvanizing layer formed on the surface of said base steel sheet, wherein in a range of 1/8 thickness to 3/8 thickness with the position of 1/4 thickness of the sheet thickness of said base steel sheet from the surface of said base steel sheet being the center, a structure of said base steel sheet is turned into a structure in which 40 to 90% in volume fraction of a ferrite phase is contained, a retained austenite phase is 5% or less in volume fraction, and further a ratio of non-recrystallized ferrite to the entire ferrite phase is 50% or less in volume fraction, a grain diameter ratio of crystal grains in the ferrite phase in said base steel sheet is 0.75 to 1.33, wherein said grain diameter ratio is defined as the ratio of an average grain diameter in the rolling direction divided by an average grain diameter in the sheet width direction of said phase, and a length ratio of hard structures dispersed in island shapes in the ferrite phase is 0.75 to 1.33, wherein said length ratio is defined as an average length in the rolling direction divided by an average length in the sheet width direction of said structures, and wherein the hard structures dispersed in island shapes are structures in which plural crystal grains composed of phases harder than ferrite aggregate to form an island shape to be dispersed in a parent phase made of a ferrite phase, and further an average aspect ratio of inclusions contained in said base steel sheet is 1.0 to 5.0, and a surface layer of said base steel sheet is turned into a decarburized layer having a thickness of 0.01 to 10.0 μm, and further an average grain diameter of oxides in the decarburized layer is 30 to 500 nm, and an average density of the oxides in the decarburized layer is in a range of 1.0 × 10 12 oxides/m 2 to 1.0 × 10 16 oxides/m 2. a base steel sheet made of steel containing: in mass%, C: 0.075 to 0.400%; Si: 0.01 to 2.00%; Mn: 0.80 to 3.50%; P: 0.0001 to 0.100%; S: 0.0001 to 0.0100%; Al: 0.001 to 2.00%; O: 0.0001 to 0.0100%; N: 0.0001 to 0.0100%; and optionally, in mass%, one kind or two or more kinds selected from Cr: 0.01 to 2.00%, Ni: 0.01 to 2.00%, Cu: 0.01 to 2.00%, Mo: 0.01 to 2.00%, B: 0.0001 to 0.0100%, W: 0.01 to 2.00%, Ti: 0.001 to 0.150%, Nb: 0.001 to 0.100%, V: 0.001 to 0.300%, 0.0001 to 0.0100 mass% in total of one kind or two or more kinds selected from Ca, Ce, Mg, Zr, La, and REM, and a balance being composed of Fe and inevitable impurities; and a hot-dip galvanizing layer formed on the surface of said base steel sheet, wherein in mass%, C: 0.075 to 0.400%; Si: 0.01 to 2.00%; Mn: 0.80 to 3.50%; P: 0.0001 to 0.100%; S: 0.0001 to 0.0100%; Al: 0.001 to 2.00%; O: 0.0001 to 0.0100%; N: 0.0001 to 0.0100%; and optionally, in mass%, one kind or two or more kinds selected from Cr: 0.01 to 2.00%, Ni: 0.01 to 2.00%, Cu: 0.01 to 2.00%, Mo: 0.01 to 2.00%, B: 0.0001 to 0.0100%, W: 0.01 to 2.00%, Ti: 0.001 to 0.150%, Nb: 0.001 to 0.100%, V: 0.001 to 0.300%, 0.0001 to 0.0100 mass% in total of one kind or two or more kinds selected from Ca, Ce, Mg, Zr, La, and REM, and a balance being composed of Fe and inevitable impurities; and a hot-dip galvanizing layer formed on the surface of said base steel sheet, wherein in a range of 1/8 thickness to 3/8 thickness with the position of 1/4 thickness of the sheet thickness of said base steel sheet from the surface of said base steel sheet being the center, a structure of said base steel sheet is turned into a structure in which 40 to 90% in volume fraction of a ferrite phase is contained, a retained austenite phase is 5% or less in volume fraction, and further a ratio of non-recrystallized ferrite to the entire ferrite phase is 50% or less in volume fraction, a grain diameter ratio of crystal grains in the ferrite phase in said base steel sheet is 0.75 to 1.33, wherein said grain diameter ratio is defined as the ratio of an average grain diameter in the rolling direction divided by an average grain diameter in the sheet width direction of said phase, and a length ratio of hard structures dispersed in island shapes in the ferrite phase is 0.75 to 1.33, wherein said length ratio is defined as an average length in the rolling direction divided by an average length in the sheet width direction of said structures, and wherein the hard structures dispersed in island shapes are structures in which plural crystal grains composed of phases harder than ferrite aggregate to form an island shape to be dispersed in a parent phase made of a ferrite phase, and further an average aspect ratio of inclusions contained in said base steel sheet is 1.0 to 5.0, and a surface layer of said base steel sheet is turned into a decarburized layer having a thickness of 0.01 to 10.0 μm, and further an average grain diameter of oxides in the decarburized layer is 30 to 500 nm, and an average density of the oxides in the decarburized layer is in a range of 1.0 × 10 12 oxides/m 2 to 1.0 × 10 16 oxides/m 2.