A three-dimensional memory device, comprising: a first silicon substrate; a first semiconductor device above the first silicon substrate, wherein the first semiconductor device includes a peripheral device layer; a peripheral interconnect layer above the peripheral device layer on the first silicon substrate, wherein the peripheral interconnect layer includes one or more interlayer dielectric layers and interconnects therein; a first single-crystal silicon layer above the peripheral interconnect layer; a first memory stack comprising a first plurality of interleaved conductor layers and dielectric layers above the first single-crystal silicon layer; a first channel structure extending vertically through the first memory stack, the first channel structure comprising a first lower plug extending into the first single-crystal silicon layer and comprising single-crystal silicon; a second single-crystal silicon layer above an array interconnect layer being configured to provide a base for a second memory stack; wherein the second single-crystal silicon layer and array interconnect layer formed thereon and the first memory stack are bonded in a face-to-face manner using hybrid bonding, wherein the array interconnect layer comprises bit lines; a first bit line is electrically connected to the first channel structure, and a through array contact (TAC, 528) extending vertically through first memory stack and first single-crystal silicon layer, wherein the through array contact (TAC, 528) is in contact with the interconnects in peripheral interconnect layer, wherein the peripheral device layer is vertically between the first silicon substrate and the first single-crystal silicon layer and electrically connected to the first bit line. a first silicon substrate; a first semiconductor device above the first silicon substrate, wherein the first semiconductor device includes a peripheral device layer; a peripheral interconnect layer above the peripheral device layer on the first silicon substrate, wherein the peripheral interconnect layer includes one or more interlayer dielectric layers and interconnects therein; a first single-crystal silicon layer above the peripheral interconnect layer; a first memory stack comprising a first plurality of interleaved conductor layers and dielectric layers above the first single-crystal silicon layer; a first channel structure extending vertically through the first memory stack, the first channel structure comprising a first lower plug extending into the first single-crystal silicon layer and comprising single-crystal silicon; a second single-crystal silicon layer above an array interconnect layer being configured to provide a base for a second memory stack; wherein the second single-crystal silicon layer and array interconnect layer formed thereon and the first memory stack are bonded in a face-to-face manner using hybrid bonding, wherein the array interconnect layer comprises bit lines; a first bit line is electrically connected to the first channel structure, and a through array contact (TAC, 528) extending vertically through first memory stack and first single-crystal silicon layer, wherein the through array contact (TAC, 528) is in contact with the interconnects in peripheral interconnect layer, wherein the peripheral device layer is vertically between the first silicon substrate and the first single-crystal silicon layer and electrically connected to the first bit line.