A banknote handling machine that manages depositing and dispensing of a banknote, the banknote handling machine comprising: a recognizing unit for recognizing a denomination being a kind of the banknote and a category being a classification based on a feature quantity of the banknote; a storage unit (72a to 72c) that stores the banknote inside thereof, and includes a memory that stores therein number-of-sheets information for the banknote stored in the storage unit (72a to 72c) by denomination and by category recognized by the recognizing unit; and a memory unit for storing therein number-of-sheets information for banknotes which are sorted and stored into each storage unit (72a to 72c), by denomination and by category recognized by the recognizing unit, wherein the category is being classified into a first category as a genuine and fit note, a second category as a genuine and unfit note, a third category as a suspect note, a fourth category as a counterfeit note, and a fifth category as a rejected note, based on a combination of a first feature quantity indicating a shape of the banknote as an optical feature quantity obtained by optically measuring the banknote, a second feature quantity indicating an amount or a wavelength of transmissive light or reflected light through/by the banknote, and a third feature quantity indicating a magnetic content contained in the banknote as a magnetic feature quantity obtained by magnetically measuring the banknote or based on a combination of a first feature quantity indicating corner-fold or an amount of a missing part of the banknote as an optical feature quantity obtained by optically measuring the banknote, a second feature quantity indicating an attenuation of infrared rays of transmissive light or reflected light through/by the banknote, and a third feature quantity indicating a magnetic content contained in a serial-number printed portion, a portrait portion, and a denomination printed portion on the banknote, as a magnetic feature quantity obtained by magnetically measuring the banknote, characterized in that the memory stores, when the suspect note or the counterfeit note is taken in the storage unit (72a to 72c), information for identifying a user of the taken-in banknote. a recognizing unit for recognizing a denomination being a kind of the banknote and a category being a classification based on a feature quantity of the banknote; a storage unit (72a to 72c) that stores the banknote inside thereof, and includes a memory that stores therein number-of-sheets information for the banknote stored in the storage unit (72a to 72c) by denomination and by category recognized by the recognizing unit; and a memory unit for storing therein number-of-sheets information for banknotes which are sorted and stored into each storage unit (72a to 72c), by denomination and by category recognized by the recognizing unit, wherein the category is being classified into a first category as a genuine and fit note, a second category as a genuine and unfit note, a third category as a suspect note, a fourth category as a counterfeit note, and a fifth category as a rejected note, a first category as a genuine and fit note, a second category as a genuine and unfit note, a third category as a suspect note, a fourth category as a counterfeit note, and a fifth category as a rejected note, based on a combination of a first feature quantity indicating a shape of the banknote as an optical feature quantity obtained by optically measuring the banknote, a second feature quantity indicating an amount or a wavelength of transmissive light or reflected light through/by the banknote, and a third feature quantity indicating a magnetic content contained in the banknote as a magnetic feature quantity obtained by magnetically measuring the banknote or based on a combination of a first feature quantity indicating corner-fold or an amount of a missing part of the banknote as an optical feature quantity obtained by optically measuring the banknote, a second feature quantity indicating an attenuation of infrared rays of transmissive light or reflected light through/by the banknote, and a third feature quantity indicating a magnetic content contained in a serial-number printed portion, a portrait portion, and a denomination printed portion on the banknote, as a magnetic feature quantity obtained by magnetically measuring the banknote, characterized in that the memory stores, when the suspect note or the counterfeit note is taken in the storage unit (72a to 72c), information for identifying a user of the taken-in banknote.