A sound processing apparatus comprising: a gain calculating unit configured to determine output gains of sounds to be output from four sound outputting units (SP1, SP2, SP3, SP4) arranged to surround a sound image localization position, the sound image localization position being a target position for a sound image, the output gains being used for fixing the sound image at the sound image localization position; and a gain adjusting unit configured to perform gain adjustment on sounds to be output from the sound outputting units (SP1, SP2, SP3, SP4) based on the output gains, characterized in that the gain calculating unit includes: a three-dimensional gain calculating unit configured to calculate output gains of a virtual sound outputting unit (VSP') and two of the sound outputting units (SP1, SP2, SP3, SP4) based on a positional relationship among the virtual sound outputting unit (VSP'), the two of the sound outputting units (SP1, SP2, SP3, SP4), and the sound image localization position; a two-dimensional gain calculating unit configured to calculate gains of other two of the sound outputting units (SP1, SP2, SP3, SP4) than the two of the sound outputting units (SP1, SP2, SP3, SP4) based on a positional relationship among the other two of the sound outputting units (SP1, SP2, SP3, SP4) and the virtual sound outputting unit (VSP'), the gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) being used for fixing a sound image at a position of the virtual sound outputting unit (VSP'); and a calculating unit (64, 65) configured to calculate output gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) based on the gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) and the output gain of the virtual sound outputting unit (VSP'); and wherein at least four of the output gains each have a value other than 0. a gain calculating unit configured to determine output gains of sounds to be output from four sound outputting units (SP1, SP2, SP3, SP4) arranged to surround a sound image localization position, the sound image localization position being a target position for a sound image, the output gains being used for fixing the sound image at the sound image localization position; and a gain adjusting unit configured to perform gain adjustment on sounds to be output from the sound outputting units (SP1, SP2, SP3, SP4) based on the output gains, characterized in that the gain calculating unit includes: a three-dimensional gain calculating unit configured to calculate output gains of a virtual sound outputting unit (VSP') and two of the sound outputting units (SP1, SP2, SP3, SP4) based on a positional relationship among the virtual sound outputting unit (VSP'), the two of the sound outputting units (SP1, SP2, SP3, SP4), and the sound image localization position; a two-dimensional gain calculating unit configured to calculate gains of other two of the sound outputting units (SP1, SP2, SP3, SP4) than the two of the sound outputting units (SP1, SP2, SP3, SP4) based on a positional relationship among the other two of the sound outputting units (SP1, SP2, SP3, SP4) and the virtual sound outputting unit (VSP'), the gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) being used for fixing a sound image at a position of the virtual sound outputting unit (VSP'); and a calculating unit (64, 65) configured to calculate output gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) based on the gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) and the output gain of the virtual sound outputting unit (VSP'); and a three-dimensional gain calculating unit configured to calculate output gains of a virtual sound outputting unit (VSP') and two of the sound outputting units (SP1, SP2, SP3, SP4) based on a positional relationship among the virtual sound outputting unit (VSP'), the two of the sound outputting units (SP1, SP2, SP3, SP4), and the sound image localization position; a two-dimensional gain calculating unit configured to calculate gains of other two of the sound outputting units (SP1, SP2, SP3, SP4) than the two of the sound outputting units (SP1, SP2, SP3, SP4) based on a positional relationship among the other two of the sound outputting units (SP1, SP2, SP3, SP4) and the virtual sound outputting unit (VSP'), the gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) being used for fixing a sound image at a position of the virtual sound outputting unit (VSP'); and a calculating unit (64, 65) configured to calculate output gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) based on the gains of the other two of the sound outputting units (SP1, SP2, SP3, SP4) and the output gain of the virtual sound outputting unit (VSP'); and wherein at least four of the output gains each have a value other than 0.