A sound signal processing device comprising: a directional point detecting unit configured to detect a directional point indicating a direction of arrival of a sound signal for each block formed through division by a predetermined time; and a directional point tracking unit configured to connect the directional points among a plurality of blocks, and detect a segment during which sound is emitted, wherein: the directional point detecting unit includes: a null beam pattern generating unit configured to generate a null beam pattern indicating directional characteristics with a relatively low sensitivity to the direction of arrival, the null beam pattern being a plot of a correspondence relationship between direction and sensitivity; a directional point detection executing unit configured to detect a local minimum point of the null beam pattern as the direction of arrival; a null beam pattern averaging unit configured to calculate an average null beam pattern, the average null beam pattern being the average of a plurality of null beam patterns at a time when a non-speech-like signal is input; and a dynamic threshold calculating unit configured to calculate a dynamic threshold as a threshold to be used in detecting the local minimum point corresponding to the direction of arrival from the null beam pattern, the dynamic threshold being set at a slightly lower value than the average null beam pattern; and the directional point detection executing unit is configured to select the local minimum point not greater than the dynamic threshold from the null beam pattern, and detects a direction corresponding to the selected local minimum point as the direction of arrival. a directional point detecting unit configured to detect a directional point indicating a direction of arrival of a sound signal for each block formed through division by a predetermined time; and a directional point tracking unit configured to connect the directional points among a plurality of blocks, and detect a segment during which sound is emitted, the directional point detecting unit includes: a null beam pattern generating unit configured to generate a null beam pattern indicating directional characteristics with a relatively low sensitivity to the direction of arrival, the null beam pattern being a plot of a correspondence relationship between direction and sensitivity; a directional point detection executing unit configured to detect a local minimum point of the null beam pattern as the direction of arrival; a null beam pattern averaging unit configured to calculate an average null beam pattern, the average null beam pattern being the average of a plurality of null beam patterns at a time when a non-speech-like signal is input; and a dynamic threshold calculating unit configured to calculate a dynamic threshold as a threshold to be used in detecting the local minimum point corresponding to the direction of arrival from the null beam pattern, the dynamic threshold being set at a slightly lower value than the average null beam pattern; and a null beam pattern generating unit configured to generate a null beam pattern indicating directional characteristics with a relatively low sensitivity to the direction of arrival, the null beam pattern being a plot of a correspondence relationship between direction and sensitivity; a directional point detection executing unit configured to detect a local minimum point of the null beam pattern as the direction of arrival; a null beam pattern averaging unit configured to calculate an average null beam pattern, the average null beam pattern being the average of a plurality of null beam patterns at a time when a non-speech-like signal is input; and a dynamic threshold calculating unit configured to calculate a dynamic threshold as a threshold to be used in detecting the local minimum point corresponding to the direction of arrival from the null beam pattern, the dynamic threshold being set at a slightly lower value than the average null beam pattern; and the directional point detection executing unit is configured to select the local minimum point not greater than the dynamic threshold from the null beam pattern, and detects a direction corresponding to the selected local minimum point as the direction of arrival.