A memory device, comprising: an alternating stack of insulating layers and electrically conductive layers located over a substrate; an array of memory stack structures extending through the alternating stack, wherein each of the memory stack structures comprises charge storage regions and a vertical semiconductor channel, and the electrically conductive layers comprise word lines for the memory stack structures; a word line decoder circuitry including switches for activating a respective word line for the memory stack structures, and located underneath the array of memory stack structures and above the substrate; a word line vertical interconnection region including multiple sets of at least one conductive interconnection structure, each set of at least one conductive interconnection structure electrically contacting a node of a respective device in the word line decoder circuitry; bit lines electrically connected to the vertical semiconductor channels through respective drain regions and extending over the array of memory stack structures; upper-interconnect-level word line connectors extending parallel to the bit lines over a portion of the array of memory stack structures, and electrically connecting a respective set of at least one conductive interconnection structure to the electrically conductive layers; at least one dielectric material layer overlying the word line decoder circuitry; a semiconductor material layer overlying the at least one dielectric material layer and underlying the alternating stack; wherein the array of memory stack structures comprises multiple blocks of memory stack structures that are laterally spaced apart from one another by through-stack contact via structures extending through the alternating stack to a top surface of the semiconductor material layer; each block of memory stack structures comprises a plurality of clusters of memory stack structures laterally spaced by word line contact via structures contacting a respective electrically conductive layer from above; the plurality of clusters of memory stack structures are in a one-dimensional array extending along a horizontal direction parallel to a lengthwise direction of the through-stack contact via structures; and wherein each word line contact via structure extending through a same block of memory stack structures contacts a different electrically conductive layer in the alternating stack. an alternating stack of insulating layers and electrically conductive layers located over a substrate; an array of memory stack structures extending through the alternating stack, wherein each of the memory stack structures comprises charge storage regions and a vertical semiconductor channel, and the electrically conductive layers comprise word lines for the memory stack structures; a word line decoder circuitry including switches for activating a respective word line for the memory stack structures, and located underneath the array of memory stack structures and above the substrate; a word line vertical interconnection region including multiple sets of at least one conductive interconnection structure, each set of at least one conductive interconnection structure electrically contacting a node of a respective device in the word line decoder circuitry; bit lines electrically connected to the vertical semiconductor channels through respective drain regions and extending over the array of memory stack structures; upper-interconnect-level word line connectors extending parallel to the bit lines over a portion of the array of memory stack structures, and electrically connecting a respective set of at least one conductive interconnection structure to the electrically conductive layers; at least one dielectric material layer overlying the word line decoder circuitry; a semiconductor material layer overlying the at least one dielectric material layer and underlying the alternating stack; wherein the array of memory stack structures comprises multiple blocks of memory stack structures that are laterally spaced apart from one another by through-stack contact via structures extending through the alternating stack to a top surface of the semiconductor material layer; each block of memory stack structures comprises a plurality of clusters of memory stack structures laterally spaced by word line contact via structures contacting a respective electrically conductive layer from above; the plurality of clusters of memory stack structures are in a one-dimensional array extending along a horizontal direction parallel to a lengthwise direction of the through-stack contact via structures; and wherein each word line contact via structure extending through a same block of memory stack structures contacts a different electrically conductive layer in the alternating stack.