A method of manufacturing a semiconductor device, comprising the steps of: forming a first columnar semiconductor layer in a portion of a substrate, and forming a first flat semiconductor layer in an upper portion of the substrate lying under the first columnar semiconductor layer; forming a first semiconductor layer of a second conductive type in a lower portion of the first columnar semiconductor layer and an entirety or an upper portion of the first flat semiconductor layer; forming a first insulating film around a lower sidewall of the first columnar semiconductor layer and above the first flat semiconductor layer; forming a gate insulating film and a gate electrode around the first columnar semiconductor layer; wherein the first insulating film has a thickness larger than that of the gate insulating film formed around the first columnar semiconductor layer and is formed between the gate electrode and the first semiconductor layer; forming a third nitride film on the first columnar semiconductor layer, on the first semiconductor layer and on the gate electrode, etching back the third nitride film in such a manner that the respective upper surfaces of the first semiconductor layer of the second conductive type, of the first columnar semiconductor layer and of the gate electrode are exposed and two insulating film sidewalls (133, 134) are thereby formed, wherein a first insulating film sidewall is formed to surround an upper sidewall of the first columnar semiconductor layer not covered by the gate insulating film and located above the gate electrode while contacting a top surface of the gate electrode and a second insulating film sidewall is formed to surround a sidewall of the gate electrode and the first insulating film; forming a second semiconductor layer of the second conductive type in an upper portion of the first columnar semiconductor layer, and forming a semiconductor layer of a first conductive type between the first semiconductor layer of the second conductive type and the second semiconductor layer of the second conductive type; and forming a metal-semiconductor compound (151, 152, 153) on each of an upper surface of the gate electrode, an upper surface of the first semiconductor layer of the second conductive type and an upper surface of the second semiconductor layer of the second conductive type. forming a first columnar semiconductor layer in a portion of a substrate, and forming a first flat semiconductor layer in an upper portion of the substrate lying under the first columnar semiconductor layer; forming a first semiconductor layer of a second conductive type in a lower portion of the first columnar semiconductor layer and an entirety or an upper portion of the first flat semiconductor layer; forming a first insulating film around a lower sidewall of the first columnar semiconductor layer and above the first flat semiconductor layer; forming a gate insulating film and a gate electrode around the first columnar semiconductor layer; wherein the first insulating film has a thickness larger than that of the gate insulating film formed around the first columnar semiconductor layer and is formed between the gate electrode and the first semiconductor layer; forming a third nitride film on the first columnar semiconductor layer, on the first semiconductor layer and on the gate electrode, etching back the third nitride film in such a manner that the respective upper surfaces of the first semiconductor layer of the second conductive type, of the first columnar semiconductor layer and of the gate electrode are exposed and two insulating film sidewalls (133, 134) are thereby formed, wherein a first insulating film sidewall is formed to surround an upper sidewall of the first columnar semiconductor layer not covered by the gate insulating film and located above the gate electrode while contacting a top surface of the gate electrode and a second insulating film sidewall is formed to surround a sidewall of the gate electrode and the first insulating film; forming a second semiconductor layer of the second conductive type in an upper portion of the first columnar semiconductor layer, and forming a semiconductor layer of a first conductive type between the first semiconductor layer of the second conductive type and the second semiconductor layer of the second conductive type; and forming a metal-semiconductor compound (151, 152, 153) on each of an upper surface of the gate electrode, an upper surface of the first semiconductor layer of the second conductive type and an upper surface of the second semiconductor layer of the second conductive type.