A three-dimensional memory device (100, 200), comprising: a substrate; a lower deck formed over the substrate, the lower deck comprising a first channel structure that vertically extends from a surface of the substrate and a first memory cell stack that has a plurality of first layers (122a - f) and second layers (120a - e), the first channel structure passing through the first memory cell stack and having sidewalls and a bottom portion, and the first layers (122a - f) and the second layers (120a - e) being alternatively stacked along the sidewalls of the first channel structure; an inter deck contact formed over a top surface of the first memory cell stack and connected with the first channel structure; and an upper deck formed over the inter deck contact, the upper deck comprising a second channel structure and a second memory cell stack, the second memory cell stack having a plurality of third layers (124a-c) and a plurality of fourth layers (126a-b), the third layers (124a-c) and the fourth layers (126a-b) being alternatively stacked along sidewalls of the second channel structure, the second channel structure passing through the second memory cell stack and having sidewalls and a bottom portion, wherein the second channel structure is vertically extending into a recess in the inter deck contact and further is extending laterally into the second memory cell stack at an interface of the inter deck contact and the second memory cell stack, and the second channel structure having a channel dielectric region located above a top surface of the inter deck contact, wherein the second channel structure comprises: a second channel layer; a second tunneling layer formed on the second channel layer and having a bottom surface; a second charge storage layer formed on the second tunneling layer and having a bottom surface; and a second blocking layer formed on the second charge storage layer and having a bottom surface; wherein the second channel layer is in contact with an upper surface of the inter deck contact and a lowermost layer of the second memory cell stack, characterized in that the second channel layer is in contact with the bottom surfaces of the second tunneling layer, second charge storage layer and second blocking layer. a substrate; a lower deck formed over the substrate, the lower deck comprising a first channel structure that vertically extends from a surface of the substrate and a first memory cell stack that has a plurality of first layers (122a - f) and second layers (120a - e), the first channel structure passing through the first memory cell stack and having sidewalls and a bottom portion, and the first layers (122a - f) and the second layers (120a - e) being alternatively stacked along the sidewalls of the first channel structure; an inter deck contact formed over a top surface of the first memory cell stack and connected with the first channel structure; and an upper deck formed over the inter deck contact, the upper deck comprising a second channel structure and a second memory cell stack, the second memory cell stack having a plurality of third layers (124a-c) and a plurality of fourth layers (126a-b), the third layers (124a-c) and the fourth layers (126a-b) being alternatively stacked along sidewalls of the second channel structure, the second channel structure passing through the second memory cell stack and having sidewalls and a bottom portion, wherein the second channel structure is vertically extending into a recess in the inter deck contact and further is extending laterally into the second memory cell stack at an interface of the inter deck contact and the second memory cell stack, and the second channel structure having a channel dielectric region located above a top surface of the inter deck contact, wherein the second channel structure comprises: a second channel layer; a second tunneling layer formed on the second channel layer and having a bottom surface; a second charge storage layer formed on the second tunneling layer and having a bottom surface; and a second blocking layer formed on the second charge storage layer and having a bottom surface; wherein the second channel layer is in contact with an upper surface of the inter deck contact and a lowermost layer of the second memory cell stack, characterized in that the second channel layer is in contact with the bottom surfaces of the second tunneling layer, second charge storage layer and second blocking layer.