A method of fabricating a memory device, comprising: forming an alternating stack of insulator layers and spacer material layers over a single crystalline semiconductor surface of a substrate; forming stepped surfaces by patterning the alternating stack, wherein the single crystalline semiconductor surface is exposed in a peripheral region from which all layers of the alternating stack are removed; after forming the stepped surfaces, forming an epitaxial semiconductor pedestal and a dielectric material portion (65, 65L) over a semiconductor surface of the semiconductor substrate in the peripheral region from which all layers of the alternating stack are removed and over the stepped surfaces, respectively, wherein the epitaxial semiconductor pedestal is in epitaxial alignment with the single crystalline semiconductor surface of the substrate; forming an array of memory stack structures through a remaining portion of the alternating stack; and forming at least one semiconductor device on the epitaxial semiconductor pedestal; the method being characterized in that the epitaxial semiconductor pedestal is formed by a selective epitaxy process prior to formation of the dielectric material portion; the method further characterized by: forming a planarization stopping layer over the alternating stack; patterning the planarization stopping layer prior to forming the stepped surfaces; depositing a dielectric material over the patterned planarization stopping layer, the stepped surfaces and the epitaxial semiconductor pedestal; and planarizing the dielectric material using the epitaxial semiconductor pedestal and the planarization stopping layer as stopping structures to form the dielectric material portion. forming an alternating stack of insulator layers and spacer material layers over a single crystalline semiconductor surface of a substrate; forming stepped surfaces by patterning the alternating stack, wherein the single crystalline semiconductor surface is exposed in a peripheral region from which all layers of the alternating stack are removed; after forming the stepped surfaces, forming an epitaxial semiconductor pedestal and a dielectric material portion (65, 65L) over a semiconductor surface of the semiconductor substrate in the peripheral region from which all layers of the alternating stack are removed and over the stepped surfaces, respectively, wherein the epitaxial semiconductor pedestal is in epitaxial alignment with the single crystalline semiconductor surface of the substrate; forming an array of memory stack structures through a remaining portion of the alternating stack; and forming at least one semiconductor device on the epitaxial semiconductor pedestal; the method being characterized in that the epitaxial semiconductor pedestal is formed by a selective epitaxy process prior to formation of the dielectric material portion; the method further characterized by: forming a planarization stopping layer over the alternating stack; patterning the planarization stopping layer prior to forming the stepped surfaces; depositing a dielectric material over the patterned planarization stopping layer, the stepped surfaces and the epitaxial semiconductor pedestal; and planarizing the dielectric material using the epitaxial semiconductor pedestal and the planarization stopping layer as stopping structures to form the dielectric material portion. forming a planarization stopping layer over the alternating stack; patterning the planarization stopping layer prior to forming the stepped surfaces; depositing a dielectric material over the patterned planarization stopping layer, the stepped surfaces and the epitaxial semiconductor pedestal; and planarizing the dielectric material using the epitaxial semiconductor pedestal and the planarization stopping layer as stopping structures to form the dielectric material portion.