A high-strength hot-dip galvanized steel sheet excellent in impact resistance property, comprising a hot-dip galvanized layer formed on a surface of a base steel sheet containing, by mass%, C: 0.075 to 0.400%, Si: 0.01 to 2.00%, Mn: 0.80 to 3.50%, P: 0.0001 to 0.100%, S: 0.0001 to 0.0100%, Al: 0.001 to 2.00%, O: 0.0001 to 0.0100%, N: 0.0001 to 0.0100%, and optionally, by mass%, one or two or more selected from Ti: 0.001 to 0.150%, Nb: 0.001 to 0.100%, V: 0.001 to 0.300%, Cr: 0.01 to 2.00%, Ni: 0.01 to 2.00%, Cu: 0.01 to 2.00%, Mo: 0.01 to 2.00%, B: 0.0001 to 0.0100%, W: 0.01 to 2.00%, 0.0001 to 0.0100% in total of one or two or more selected from Ca, Ce, Mg, Zr, La, and REM, and a balance composed of Fe and inevitable impurities, wherein: the base steel sheet has a steel sheet structure in a range of 1/8 thickness to 3/8 thickness centered around 1/4 thickness of a sheet thickness from a surface, in which a volume fraction of a retained austenite phase is 5% or less, and a total volume fraction of a bainite phase, a bainitic ferrite phase, a fresh martensite phase and a tempered martensite phase is 40% or more; an average effective crystal grain diameter and a maximum effective crystal grain diameter in the range of 1/8 thickness to 3/8 thickness centered around 1/4 thickness of the sheet thickness from the surface are 5.0 μm or less and 20 μm or less, respectively; and a decarburized layer with a thickness of 0.01 μm to 10.0 μm is formed on a surface layer portion, in which a density of oxides dispersed in the decarburized layer is 1.0 × 10 12 to 1.0 × 10 oxides/m 2, and an average grain diameter of the oxides is 500 nm or less; and wherein the average effective crystal grain diameter is determined by an intercept method, wherein a grain boundary map is created by using the effective crystal grain boundary, lines, whose lengths are 300 μm or more in total, parallel to the rolling direction are written on the grain boundary map, and a value obtained by dividing the total lengths of the lines by a number of intersection points of the lines and the effective crystal grain boundary is set to the average effective crystal grain diameter; and wherein a grain diameter at a position, at which a distance between adjacent intersection points is the largest, is set to the maximum effective crystal grain diameter. C: 0.075 to 0.400%, Si: 0.01 to 2.00%, Mn: 0.80 to 3.50%, P: 0.0001 to 0.100%, S: 0.0001 to 0.0100%, Al: 0.001 to 2.00%, O: 0.0001 to 0.0100%, N: 0.0001 to 0.0100%, and Ti: 0.001 to 0.150%, Nb: 0.001 to 0.100%, V: 0.001 to 0.300%, Cr: 0.01 to 2.00%, Ni: 0.01 to 2.00%, Cu: 0.01 to 2.00%, Mo: 0.01 to 2.00%, B: 0.0001 to 0.0100%, W: 0.01 to 2.00%, 0.0001 to 0.0100% in total of one or two or more selected from Ca, Ce, Mg, Zr, La, and REM, and the base steel sheet has a steel sheet structure in a range of 1/8 thickness to 3/8 thickness centered around 1/4 thickness of a sheet thickness from a surface, in which a volume fraction of a retained austenite phase is 5% or less, and a total volume fraction of a bainite phase, a bainitic ferrite phase, a fresh martensite phase and a tempered martensite phase is 40% or more; an average effective crystal grain diameter and a maximum effective crystal grain diameter in the range of 1/8 thickness to 3/8 thickness centered around 1/4 thickness of the sheet thickness from the surface are 5.0 μm or less and 20 μm or less, respectively; and a decarburized layer with a thickness of 0.01 μm to 10.0 μm is formed on a surface layer portion, in which a density of oxides dispersed in the decarburized layer is 1.0 × 10 12 to 1.0 × 10 oxides/m 2, and an average grain diameter of the oxides is 500 nm or less; and wherein the average effective crystal grain diameter is determined by an intercept method, wherein a grain boundary map is created by using the effective crystal grain boundary, lines, whose lengths are 300 μm or more in total, parallel to the rolling direction are written on the grain boundary map, and a value obtained by dividing the total lengths of the lines by a number of intersection points of the lines and the effective crystal grain boundary is set to the average effective crystal grain diameter; and wherein a grain diameter at a position, at which a distance between adjacent intersection points is the largest, is set to the maximum effective crystal grain diameter.