An analog neuromorphic memory system comprising: a vector-by-matrix multiplication array comprising an array of non-volatile memory cells organized into rows and columns, wherein each column is connected to a bit line and each memory cell comprises a word line terminal and a source line terminal ; a word line decoder circuit coupled to the word line terminals of the non-volatile memory cells through word lines, wherein the word line decoder circuit is capable of applying a low voltage or a high voltage to coupled word line terminals, the word line decoder circuit comprising an NMOS transistor for pulling the word line to the low voltage and a PMOS transistor for pulling the word line to the high voltage, a gate of the NMOS transistor and a gate of the PMOS transistor coupled to a high voltage control signal (HVO_B), and wherein the word lines are parallel to the bit lines; a source line decoder circuit coupled to the source line terminals of the non-volatile memory cells through source lines, wherein the source line decoder circuit is capable of applying a low voltage or a high voltage to coupled source line terminals, the source line decoder circuit comprising a first NMOS transistor for pulling the source line to the low voltage, a second NMOS transistor for pulling the source line to the high voltage, and a third NMOS transistor for monitoring a voltage of the source line, a gate of the first NMOS transistor coupled to the high voltage control signal (HVO_B) and a gate of the second NMOS transistor and a gate of the third NMOS transistor coupled to a complement of the high voltage control signal (HVO), and wherein the source lines are perpendicular to the bit lines and the word lines; and a high voltage level shifter for generating the high voltage control signal (HVO_B) and the complement of the high voltage control signal (HVO) in response to an enable signal (EN). a vector-by-matrix multiplication array comprising an array of non-volatile memory cells organized into rows and columns, wherein each column is connected to a bit line and each memory cell comprises a word line terminal and a source line terminal ; a word line decoder circuit coupled to the word line terminals of the non-volatile memory cells through word lines, wherein the word line decoder circuit is capable of applying a low voltage or a high voltage to coupled word line terminals, the word line decoder circuit comprising an NMOS transistor for pulling the word line to the low voltage and a PMOS transistor for pulling the word line to the high voltage, a gate of the NMOS transistor and a gate of the PMOS transistor coupled to a high voltage control signal (HVO_B), and wherein the word lines are parallel to the bit lines; a source line decoder circuit coupled to the source line terminals of the non-volatile memory cells through source lines, wherein the source line decoder circuit is capable of applying a low voltage or a high voltage to coupled source line terminals, the source line decoder circuit comprising a first NMOS transistor for pulling the source line to the low voltage, a second NMOS transistor for pulling the source line to the high voltage, and a third NMOS transistor for monitoring a voltage of the source line, a gate of the first NMOS transistor coupled to the high voltage control signal (HVO_B) and a gate of the second NMOS transistor and a gate of the third NMOS transistor coupled to a complement of the high voltage control signal (HVO), and wherein the source lines are perpendicular to the bit lines and the word lines; and a high voltage level shifter for generating the high voltage control signal (HVO_B) and the complement of the high voltage control signal (HVO) in response to an enable signal (EN). <EOS>