A method for forming a three-dimensional memory device, comprising: forming an initial channel hole in a stack structure (21, 51) of a plurality first layers and a plurality of second layers alternatingly arranged over a substrate (10, 20); forming an offset between a side surface of each one of the plurality of first layers and a side surface of each one of the plurality of second layers on a sidewall of the initial channel hole to form a channel hole; forming a semiconductor channel (14, 24) by filling the channel hole with a channel-forming structure, the semiconductor channel (14, 24) having a memory layer comprising a plurality of first memory portions each surrounding a bottom of a respective second layer and a plurality of second memory portions each connecting adjacent first memory portions; removing the plurality of first layers; forming a plurality of conductor layers from the plurality of second layers; and forming a gate-to-gate dielectric layer between the adjacent conductor layers, the gate-to-gate dielectric layer comprising at least one sub-layer of silicon oxynitride and an airgap, wherein removing the plurality of first layers comprises: forming a first initial slit opening (25, 55) extending through the stack structure (21, 51) and exposing the substrate (10, 20); and removing the plurality of first layers through the first initial slit opening (25, 55) to form a plurality of lateral recesses that expose portions of the semiconductor channel (14, 24), wherein forming the plurality of conductor layers, the gate-to-gate dielectric layer, and a second initial slit opening (35A, 35B) comprise: forming a composite layer (17-1, 17-2) from a portion of each of the plurality of second layers, a remaining portion of the respective second layer forming a respective conductor layer, a pair of composite layers on the adjacent conductor layers and facing each other forming the gate-to-gate dielectric layer, forming the second initial slit opening (35A, 35B) from the first initial slit opening (25, 55), the pair of composite layers (17-1, 17-2) each having at least one sub-layer of silicon oxide and one sub-layer of silicon oxynitride, and wherein each of the sub-layers of silicon oxynitride is sandwiched by layers of silicon oxide, the airgap being between the pair of composite layers, and wherein the plurality of second layers comprise polysilicon and forming the composite layer (17-1, 17-2) comprises: performing, through the first initial slit opening (25, 55) and the plurality of lateral recesses, at least one oxidation reaction and at least one nitriding reaction on the plurality of second layers, a reacted portion of each of the plurality of second layers forming the respective composite layer (17-1, 17-2), an unreacted portion of each of the plurality of second layers forming the respective conductor layer. forming an initial channel hole in a stack structure (21, 51) of a plurality first layers and a plurality of second layers alternatingly arranged over a substrate (10, 20); forming an offset between a side surface of each one of the plurality of first layers and a side surface of each one of the plurality of second layers on a sidewall of the initial channel hole to form a channel hole; forming a semiconductor channel (14, 24) by filling the channel hole with a channel-forming structure, the semiconductor channel (14, 24) having a memory layer comprising a plurality of first memory portions each surrounding a bottom of a respective second layer and a plurality of second memory portions each connecting adjacent first memory portions; removing the plurality of first layers; forming a plurality of conductor layers from the plurality of second layers; and forming a gate-to-gate dielectric layer between the adjacent conductor layers, the gate-to-gate dielectric layer comprising at least one sub-layer of silicon oxynitride and an airgap, wherein removing the plurality of first layers comprises: forming a first initial slit opening (25, 55) extending through the stack structure (21, 51) and exposing the substrate (10, 20); and removing the plurality of first layers through the first initial slit opening (25, 55) to form a plurality of lateral recesses that expose portions of the semiconductor channel (14, 24), wherein forming the plurality of conductor layers, the gate-to-gate dielectric layer, and a second initial slit opening (35A, 35B) comprise: forming a composite layer (17-1, 17-2) from a portion of each of the plurality of second layers, a remaining portion of the respective second layer forming a respective conductor layer, a pair of composite layers on the adjacent conductor layers and facing each other forming the gate-to-gate dielectric layer, forming the second initial slit opening (35A, 35B) from the first initial slit opening (25, 55), the pair of composite layers (17-1, 17-2) each having at least one sub-layer of silicon oxide and one sub-layer of silicon oxynitride, and wherein each of the sub-layers of silicon oxynitride is sandwiched by layers of silicon oxide, the airgap being between the pair of composite layers, and wherein the plurality of second layers comprise polysilicon and forming the composite layer (17-1, 17-2) comprises: performing, through the first initial slit opening (25, 55) and the plurality of lateral recesses, at least one oxidation reaction and at least one nitriding reaction on the plurality of second layers, a reacted portion of each of the plurality of second layers forming the respective composite layer (17-1, 17-2), an unreacted portion of each of the plurality of second layers forming the respective conductor layer.