An analog neuromorphic memory system comprising: a vector-by-matrix multiplication array comprising an array of non-volatile memory cells organized into rows and columns, wherein each column is connected to a bit line and each memory cell (210, 310, 510) comprises a word line terminal and a source line terminal; a word line decoder circuit coupled to the word line terminals of the non-volatile memory cells through word lines, wherein the word line decoder circuit is capable of applying a low voltage or a high voltage to coupled word line terminals, the word line decoder circuit comprising an NMOS transistor for pulling the word line to the low voltage and a PMOS transistor for pulling the word line to the high voltage, a gate of the NMOS transistor and a gate of the PMOS transistor coupled to a high voltage control signal (HVO_B), and wherein the word lines are parallel to the bit lines; a source line decoder circuit (2804, 2907) coupled to the source line terminals of the non-volatile memory cells through source lines, wherein the source line decoder circuit is capable of applying a low voltage or a high voltage to coupled source line terminals, the source line decoder circuit comprising a first NMOS transistor for pulling the source line to the low voltage, a second NMOS transistor for pulling the source line to the high voltage, and a third NMOS transistor for monitoring a voltage of the source line, a gate of the first NMOS transistor coupled to the high voltage control signal (HVO_B) and a gate of the second NMOS transistor and a gate of the third NMOS transistor coupled to a complement of the high voltage control signal, and wherein the source lines are perpendicular to the bit lines and the word lines; and a high voltage level shifter for generating the high voltage control signal (HVO_B) and the complement of the high voltage control signal in response to an enable signal. a vector-by-matrix multiplication array comprising an array of non-volatile memory cells organized into rows and columns, wherein each column is connected to a bit line and each memory cell (210, 310, 510) comprises a word line terminal and a source line terminal; a word line decoder circuit coupled to the word line terminals of the non-volatile memory cells through word lines, wherein the word line decoder circuit is capable of applying a low voltage or a high voltage to coupled word line terminals, the word line decoder circuit comprising an NMOS transistor for pulling the word line to the low voltage and a PMOS transistor for pulling the word line to the high voltage, a gate of the NMOS transistor and a gate of the PMOS transistor coupled to a high voltage control signal (HVO_B), and wherein the word lines are parallel to the bit lines; a source line decoder circuit (2804, 2907) coupled to the source line terminals of the non-volatile memory cells through source lines, wherein the source line decoder circuit is capable of applying a low voltage or a high voltage to coupled source line terminals, the source line decoder circuit comprising a first NMOS transistor for pulling the source line to the low voltage, a second NMOS transistor for pulling the source line to the high voltage, and a third NMOS transistor for monitoring a voltage of the source line, a gate of the first NMOS transistor coupled to the high voltage control signal (HVO_B) and a gate of the second NMOS transistor and a gate of the third NMOS transistor coupled to a complement of the high voltage control signal, and wherein the source lines are perpendicular to the bit lines and the word lines; and a high voltage level shifter for generating the high voltage control signal (HVO_B) and the complement of the high voltage control signal in response to an enable signal.