A neural network device, comprising: a first plurality of synapses configured to receive a first plurality of inputs and to generate therefrom a first plurality of outputs, wherein the first plurality of synapses comprises: a plurality of memory cells, wherein each of the memory cells includes spaced apart source and drain regions (14, 16) formed in a semiconductor substrate with a channel region extending there between, a floating gate disposed over and insulated from a first portion of the channel region, a first gate disposed over and insulated from a second portion of the channel region, and a second gate (22, 30) disposed over and insulated from the floating gate or disposed over and insulated from the source region; each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate; the plurality of memory cells are configured to generate the first plurality of outputs based upon the first plurality of inputs and the stored weight values; wherein the memory cells of the first plurality of synapses are arranged in rows and columns, and wherein the first plurality of synapses comprises: a plurality of first lines each electrically connecting together the first gates in one of the rows of the memory cells; a plurality of second lines each electrically connecting together the second gates in one of the rows of the memory cells; a plurality of third lines each electrically connecting together the source regions in one of the columns of the memory cells; a plurality of fourth lines each electrically connecting together the drain regions in one of the columns of the memory cells; wherein the first plurality of synapses is configured to receive the first plurality of inputs as electrical voltages on the plurality of second lines, and to provide the first plurality of outputs as electrical currents on the plurality of third lines or on the plurality of fourth lines. a first plurality of synapses configured to receive a first plurality of inputs and to generate therefrom a first plurality of outputs, wherein the first plurality of synapses comprises: a plurality of memory cells, wherein each of the memory cells includes spaced apart source and drain regions (14, 16) formed in a semiconductor substrate with a channel region extending there between, a floating gate disposed over and insulated from a first portion of the channel region, a first gate disposed over and insulated from a second portion of the channel region, and a second gate (22, 30) disposed over and insulated from the floating gate or disposed over and insulated from the source region; each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate; the plurality of memory cells are configured to generate the first plurality of outputs based upon the first plurality of inputs and the stored weight values; a plurality of memory cells, wherein each of the memory cells includes spaced apart source and drain regions (14, 16) formed in a semiconductor substrate with a channel region extending there between, a floating gate disposed over and insulated from a first portion of the channel region, a first gate disposed over and insulated from a second portion of the channel region, and a second gate (22, 30) disposed over and insulated from the floating gate or disposed over and insulated from the source region; each of the plurality of memory cells is configured to store a weight value corresponding to a number of electrons on the floating gate; the plurality of memory cells are configured to generate the first plurality of outputs based upon the first plurality of inputs and the stored weight values; wherein the memory cells of the first plurality of synapses are arranged in rows and columns, and wherein the first plurality of synapses comprises: a plurality of first lines each electrically connecting together the first gates in one of the rows of the memory cells; a plurality of second lines each electrically connecting together the second gates in one of the rows of the memory cells; a plurality of third lines each electrically connecting together the source regions in one of the columns of the memory cells; a plurality of fourth lines each electrically connecting together the drain regions in one of the columns of the memory cells; a plurality of first lines each electrically connecting together the first gates in one of the rows of the memory cells; a plurality of second lines each electrically connecting together the second gates in one of the rows of the memory cells; a plurality of third lines each electrically connecting together the source regions in one of the columns of the memory cells; a plurality of fourth lines each electrically connecting together the drain regions in one of the columns of the memory cells; wherein the first plurality of synapses is configured to receive the first plurality of inputs as electrical voltages on the plurality of second lines, and to provide the first plurality of outputs as electrical currents on the plurality of third lines or on the plurality of fourth lines.