Audio controlling apparatus, audio correction apparatus, and audio correction method

According to one embodiment, an audio controlling apparatus includes a first receiver configured to receive audio signal, a second receiver configured to receive environmental sound, a temporary gain calculator configured to calculate temporary gain based on environmental sound received by second receiver, a sound type determination module configured to determine sound type of main component of audio signal received by first receiver, and a gain controller configured to stabilize temporary gain that is calculated by temporary gain calculator and set gain, when it is determined that sound type of main component of audio signal received by first receiver is music.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based upon and claims the benefit of priority from prior Japanese Patent Application No. 2011-141098, filed Jun. 24, 2011, the entire contents of which are incorporated herein by reference.

FIELD

Embodiments described herein relate generally to an audio controlling apparatus, an audio correction apparatus, and an audio correction method.

BACKGROUND

In prior art, audio correction apparatuses which performs correction for audio signals and thereby can achieve increase in sound quality are generally known. The audio correction apparatuses detect, for example, a surrounding background noise (environmental sound), and performs sound volume control and equalizing processing according to the frequency for the audio signal, based on the detected environmental sound.

The audio correction apparatuses calculate a gain based on the level of the environmental sound for each frequency band. The audio correction apparatuses perform correction by multiplying the audio signal by the calculated gain. Specifically, the audio correction apparatuses successively determine the gain used when the audio signal is corrected, according to the level of the environmental sound. Therefore, when an environmental sound which varies with lapse of time is generated, the audio signal which is corrected by the audio correction apparatuses may provide the user with a feeling that something is wrong.

For example, when a momentary environmental sound is generated, the audio correction apparatuses abruptly change the gain. The audio correction apparatuses abruptly correct the audio signal based on the calculated gain. As a result, there is the problem that the corrected audio signal may provide the user with a feeling that something is wrong.

DETAILED DESCRIPTION

Various embodiments will be described hereinafter with reference to the accompanying drawings. In general, according to one embodiment, an audio controlling apparatus is provided connected to an audio correction apparatus that includes an audio correction module that corrects an audio signal. The audio controlling apparatus includes a first receiver configured to receive the audio signal, a second receiver configured to receive an environmental sound, a temporary gain calculator configured to calculate a temporary gain based on the environmental sound received by the second receiver, a sound type determination module configured to determine a sound type of a main component of the audio signal received by the first receiver, a gain controller configured to stabilize the temporary gain that is calculated by the temporary gain calculator and set a gain, when it is determined that the sound type of the main component of the audio signal received by the first receiver is music, and an output module configured to output the gain that is set by the gain controller to the audio correction module of the audio correction apparatus.

An audio controlling apparatus, an audio correction apparatus, and an audio correction method according to an embodiment will be explained in detail hereinafter with reference to drawings.

FIG. 1illustrates an example of a broadcasting receiver100according to an embodiment.

In the present embodiment, suppose that the broadcasting receiver100displays an image on a liquid crystal display device based on content.

The broadcasting receiver100comprises a broadcasting input terminal101, a tuner111, a demodulator112, a signal processor113, a communication interface114, a sound processor121, a sound output terminal122, an environmental sound input terminal124, an image processor131, an OSD processor132, a display processor133, an image output terminal135, a controller150, an operation input module161, a light-receiving module162, a card connector164, a USB connector166, and a disk drive170.

The broadcasting input terminal101is an input terminal, to which digital broadcasting signals that are received by, for example, an antenna110is inputted. The antenna110receives, for example, ground-wave digital broadcasting signals, BS (broadcasting satellite) digital broadcasting signals, and/or 110° CS (communication satellite) digital broadcasting signals. Specifically, contents such as programs that are supplied by broadcasting signals are inputted to the broadcasting input terminal101.

The broadcasting input terminal101supplies the received digital broadcasting signals to the tuner111. The tuner111is a digital broadcasting signal tuner. The tuner111performs tuning (selection) of the digital broadcasting signals supplied from the antenna110. The tuner111transmits a tuned digital broadcasting signal to the demodulator112.

The demodulator112demodulates the received digital broadcasting signal. The demodulator112inputs the demodulated digital broadcasting signal (content) to the signal processor113. Specifically, the antenna110, the tuner111, and the demodulator112function as receiving means for receiving content.

The signal processor113functions as signal processing means for performing signal processing on the digital broadcasting signal (moving-image content data). The signal processor113performs signal processing on the digital broadcasting signal supplied from the demodulator112. Specifically, the signal processor113divides the digital broadcasting signal into an image signal, a sound signal, and another data signal. The signal processor113supplies the sound signal to the sound processor121. In addition, the signal processor113supplies the image signal to the image processor131. The signal processor113also supplies the data signal to the controller150and/or the OSD processor132.

The communication interface114includes an interface, such as an HDMI (High Definition Multimedia Interface), which can receive content. The communication interface114receives multiplex content, which includes a digital image signal and a digital sound signal, from another apparatus. The communication interface114supplies a digital signal (content) received from another apparatus to the signal processor113. Specifically, the communication interface114functions as receiving means for receiving content.

The signal processor113performs signal processing on the digital signal that is received from the communication interface114. For example, the signal processor113divides the digital signal into a digital image signal, a digital sound signal, and a data signal. The signal processor113supplies the digital sound signal to the sound processor121. In addition, the signal processor113supplies the digital image signal to the image processor131. The signal processor113also supplies the data signal to the controller150and/or the OSD processor132.

The signal processor113selects one of the content which is inputted to the communication interface114, and the content which is inputted to the broadcasting input terminal101, and performs signal processing on the selected content. Specifically, the signal processor113performs signal dividing processing on one of the digital broadcasting signal and the digital signal.

The sound processor121converts the digital sound signal received from the signal processor113into a signal (audio signal) of a format which can be played back by a speaker300. The sound processor121outputs the audio signal to the sound output terminal122. The sound output terminal122outputs the supplied audio signal to the outside of the apparatus. Thereby, the speaker300which is connected to the sound output terminal122plays back sound based on the supplied audio signal.

The audio signal includes various types of sounds according to the content. For example, the audio signal such as a TV program includes a signal based on a speech (speech) generated by a person, music (music), and various noises (background noise). The noise in this case includes, for example, applause, cheers, and other various noises.

The environmental sound input terminal124is a terminal to which a device which detects sound, such as a microphone500, is connected. The microphone500is provided in the vicinity of the broadcasting receiver100, detects surrounding sound (background noise) around the broadcasting receiver100, and converts the sound into a signal. The microphone500supplies the signal to the environmental sound input terminal124. The environmental sound input terminal124supplies the signal supplied from the microphone500as environmental sound to the sound processor121.

The image processor131converts an image signal received from the signal processor113into an image signal of a format which can be played back by a display400. Specifically, the image processor131decodes (plays back) the image signal received from the signal processor113into an image signal of a format which can be played back by the display400. In addition, the image processor131superposes the OSD signal which is supplied from the OSD processor132on the image signal. The image processor131outputs the image signal to the display processor133.

The OSD processor132generates an OSD signal to display a GUI (Graphic User Interface) picture, subtitles, time, and other information on the screen, based on the data signal supplied from the signal processor113, and/or the control signal supplied from the controller150.

The display processor133performs image quality control processing for color, brightness, sharpness, contrast, and other properties, on the received image signal, based on control from the controller150. The display controller133outputs the image signal which has been subjected to image quality control to the image output terminal135. Thereby, the display400which is connected to the image output terminal135displays an image based on the supplied image signal.

The display400includes, for example, a liquid crystal display device which includes a liquid crystal display panel that includes a plurality of pixels that are arranged in rows and columns, and a backlight that illuminates the liquid crystal panel. The display400displays an image based on the image signal supplied from the broadcasting receiver100.

The broadcasting receiver100may have a structure in which the display400is included in the receiver, instead of the image output terminal135. The broadcasting receiver100may have a structure in which the speaker300is included in the receiver, instead of the sound output terminal122.

The controller150functions as control means for controlling operations of the modules in the broadcasting receiver100. The controller150includes a CPU151, a ROM152, a RAM153, and an EEPROM154. The controller150performs various processing, based on an operation signal supplied from the operation input module161.

The CPU151includes an operation unit which executes various operation processing. The CPU151realizes various functions by executing programs stored in the ROM152or the EEPROM154.

The ROM152stores a program to control the broadcasting receiver100, and programs to execute various functions. The CPU151starts a program stored in the ROM152, based on an operation signal supplied from the operation input module161. Thereby, the controller150controls operations of the modules.

The RAM153functions as a work memory of the CPU151. Specifically, the RAM153stores an operation result of the CPU151, and data read by the CPU151.

The EEPROM154is a nonvolatile memory which stores various setting information items and programs.

The operation input module161is input means which includes operation keys, a keyboard, a mouse, a touch pad, or another input device which can generate an operation signal in response to an operation. For example, the operation input module161generates an operation signal in response to an operation. The operation input module161supplies the generated operation signal to the controller150.

The touch pad includes an electrostatic sensor, a thermo sensor, or a device which generates positional information based on another method. When the broadcasting receiver100includes the display400, the operation input module161may have a structure of including a touch panel which is formed as one unitary piece with the display400.

The light-receiving module162includes, for example, a sensor which receives an operation signal from a remote controller163. The light-receiving module162supplies the received operation signal to the controller150. The remote controller163generates an operation signal based on a user's operation. The remote controller163transmits the generated operation signal to the light-receiving module162by infrared communication. The light-receiving module162and the remote controller163may have a structure of performing transmission/reception of the operation signal by another wireless communication such as radio communication.

The card connector164is an interface to perform communication with, for example, a memory card165which stores moving-image content. The card connector164reads out moving-image content data from the connected memory card165, and supplies the content data to the controller150.

The USB connector166is an interface to perform communication with a USB device167. The USB connector166supplies a signal supplied from the connected USB device167to the controller150.

For example, when the USB device167is an operation input device such as a keyboard, the USB connector166receives an operation signal from the USB device167. The USB connector166supplies the received operation signal to the controller150. In this case, the controller150executes various processing, based on the operation signal supplied from the USB connector166.

In addition, for example, when the USB device167is a storage device which stores moving-image content data, the USB connector166can obtain the content from the USB device167. The USB connector166supplies the obtained content to the controller150.

The disk drive170includes a drive to which an optical disk M, such as a compact disk (CD), a digital versatile disk (DVD), a Blu-ray disk (registered trademark), and another optical disk which can store moving-image content data, can be attached. The disk drive170reads out content from the attached optical disk M, and supplies the read content to the controller150.

The broadcasting receiver100also includes a power source (not shown). The power source supplies electric power to the modules of the broadcasting receiver100. The power source converts electric power which is supplied through an AC adaptor or the like, and supplies the electric power to the modules. The power source may include a battery. In such a case, the power source charges the battery with electric power supplied through the AC adaptor or the like. The power source supplies the electric power stored in the battery to the modules of the broadcasting receiver100.

The broadcasting receiver100may also include another interface. The interface is, for example, a Serial-ATA, or a LAN port. The broadcasting receiver100can obtain and play back content which is stored in a device that is connected by the interface. The broadcasting receiver100can output a played-back audio signal and image signal to a device connected by the interface.

When the broadcasting receiver100is connected to a network through the interface, the broadcasting receiver100can obtain and play back moving-image content data on the network.

In addition, the broadcasting receiver100may include a storage device such as a hard disk (HDD), a solid state disk (SDD), and a semiconductor memory. When the storage device stores moving-image content data, the broadcasting receiver100can read out and play back the content stored in the storage device. Besides, the broadcasting receiver100can store a broadcasting signal, or content supplied by a network or the like, in the storage device.

The sound processor121includes an audio controller200. The audio controller200corrects an audio signal played back by the sound processor121. Thereby, the audio controller200controls the audio signal such that sound which can be heard more easily for the user is played back from the speaker300. For example, the audio controller200can correct the audio signal for each frequency, based on the environmental sound supplied from the environmental sound input terminal124and the playback audio signal.

FIG. 2illustrates an example of the audio controller200illustrated inFIG. 1.

As illustrated inFIG. 2, the audio controller200includes an input terminal210, an equalizer220, an output terminal230, a characteristic parameter calculator240, a score calculator250, an environmental sound analyzer260, and a gain controller280.

The audio controller200corrects the audio signal such that the audio signal is more easily heard by the user, based on the sound type of the audio signal and the characteristic of the environmental sound which is detected by the microphone500when the broadcasting receiver100plays back the audio signal. The environmental sound is a signal which is obtained by suppressing loop of sound outputted from the speaker300by an echo canceller or the like.

The input terminal210is receiving means for receiving an audio signal which is played back by the sound processor121. The input terminal210supplies the received audio signal to the equalizer220, the characteristic parameter calculator240, and the environmental sound analyzer260.

The equalizer220corrects the audio signal in accordance with a gain characteristic designated by the gain controller280. The equalizer220transmits the corrected audio signal (corrected audio signal) to the output terminal230.

The output terminal230outputs the corrected audio signal corrected by the equalizer220. When the sound output terminal122is connected with the output terminal230, the output terminal230outputs the corrected audio signal to the outside of the broadcasting receiver100through the sound output terminal122. As a result, the speaker300connected to the sound output terminal122plays back sound based on the corrected audio signal.

The characteristic parameter calculator240calculates various characteristic parameters based on the audio signal. In this case, as illustrated inFIG. 3, the characteristic parameter calculator240divides the audio signal into a plurality of frames of several hundred milliseconds. The characteristic parameter calculator240further divides each frame into a plurality of subframes of twenty or thirty milliseconds.

FIG. 4illustrates an example of operation of the audio controller200.

First, the characteristic parameter calculator240of the audio controller200performs FFT processing (Fast Fourier Transform) on the audio signal, and calculates a frequency characteristic (Step S11). In addition, the characteristic parameter calculator240calculates characteristic parameters based on the above frequency characteristic and a signal characteristic in a time domain (Step S12).

The characteristic parameter calculator240calculates various characteristic parameters for each subframe. Therefore, the characteristic parameter calculator240obtains discrimination information for generating various characteristic parameters for each subframe. The characteristic parameter calculator240calculates statistics of the obtained discrimination information for each frame, and thereby calculates characteristic parameters.

Specifically, the characteristic parameter calculator240obtains discrimination information to distinguish the speech signal and the music signal for each subframe from the audio signal. In addition, the characteristic parameter calculator240obtains discrimination information to distinguish the music signal from noise for each subframe from the audio signal. The characteristic parameter calculator240also obtains various discrimination information items to distinguish the speech and music signals from the noise signal for each subframe from the audio signal.

For example, the characteristic parameter calculator240calculates statistics (for example, average, distribution, maximum, and minimum) for each frame based on the obtained various discrimination information items. Thereby, the characteristic parameter calculator240calculates various characteristic parameters.

For example, the characteristic parameter calculator240calculates various characteristic parameters to distinguish the speech signal from the music signal, based on the discrimination information. In addition, the characteristic parameter calculator240calculates various characteristic parameters to distinguish the music signal from noise based on the discrimination information. The characteristic parameter calculator240also calculates various characteristic parameters to distinguish the speech and music signals from the noise signal based on the discrimination information.

For example, the characteristic parameter calculator240calculates a sum of squares (power value) of the signal amplitude of the audio signal for each subframe, as discrimination information. The characteristic parameter calculator240calculates statistics for each frame based on the calculated power value, and thereby generates a characteristic parameter pw relating to the power value.

The characteristic parameter calculator240also calculates a zero-cross frequency of the audio signal for each subframe as discrimination information. The zero-cross frequency is the number of times with which the audio signal crosses zero in the amplitude direction, when the horizontal axis indicates time and the vertical axis indicates amplitude. The characteristic parameter calculator240calculates statistics for each frame based on the calculated zero-cross frequency, and thereby generates a characteristic parameter zc relating to the zero-cross frequency.

In addition, the characteristic parameter calculator240calculates spectral variation in a frequency region of the audio signal for each subframe, as discrimination information. The characteristic parameter calculator240calculates statistics for each frame based on the calculated spectral variation, and thereby generates a characteristic parameter sf relating to spectral variation.

The characteristic parameter calculator240also calculates a power ratio (LR power ratio) of left and right (LR) signals of 2-channel stereo in the audio signal for each subframe, as distinction information. The characteristic parameter calculator240calculates statistics for each frame based on the calculated LR power ratio, and thereby generates a characteristic parameter lr relating to the LR power ratio.

The characteristic parameter calculator240also calculates spectral flatness of the audio signal for each subframe, as discrimination information. The characteristic parameter calculator240calculates statistics for each frame based on the calculated spectral flatness, and thereby generates a characteristic parameter SFM relating to the noise signal.

The characteristic parameter pw relating to the power value indicates a difference in signal power between subframes. For example, the characteristic parameter pw indicates distribution of the power value between subframes. When the main component of the audio signal is speech, the difference in signal power between subframes tends to be large. Specifically, when the main component of the audio signal is speech, the value of the characteristic parameter pw increases.

The characteristic parameter zc relating to the zero-cross frequency indicates variation of the number of times with which the audio signal crosses zero in the amplitude direction, when the horizontal axis indicates time and the vertical axis indicates amplitude, as described above. In the speech signal, the zero-cross frequency is high in a consonant, and the zero-cross frequency is low in a vowel. Specifically, when the main component of the audio signal is speech, distribution of the value of the characteristic parameter zc relating to the zero-cross frequency between subframes tends to increase.

The characteristic parameter sf relating to the spectral variation indicates spectral variation in the frequency region of the audio signal for each subframe, as described above. The speech signal has sharper variations in frequency characteristics than that of the music signal. Therefore, when the main component of the audio signal is speech, distribution of the characteristic parameter sf relating to the spectral variation tends to be large.

The characteristic parameter lr relating to the LR power ratio indicates a power ratio (LR power ratio) of left and right (LR) signals of 2-channel stereo in the audio signal for each subframe, as described above. In the music signal, performances of musical instruments other than the vocal are often localized other than the center. Therefore, when the main component of the audio signal is music, the power ratio between the left and right channels tends to increase.

The characteristic parameter SFM relating to the noise signal is a parameter based on the spectral flatness of the audio signal for each subframe, as described above. The noise signal tends to form a flat spectrum. Therefore, when the main component of the audio signal is noise, distribution of the audio signal between subframes tends to decrease.

The characteristic parameter calculator240supplies the generated various characteristic parameters to the score calculator250.

The score calculator250calculates a speech score SS, a music score SM, and a noise score SN, based on the above characteristic parameters (Step S13).

The score calculator250calculates a speech and music discrimination score S1which indicates whether the audio signal is close to speech or music, based on the various characteristic parameters supplied from the characteristic parameters240. The score calculator250also calculates a music and background sound discrimination score S2which indicates whether the audio signal is close to music or background noise, based on the various characteristic parameters supplied from the characteristic parameter calculator240.

The score calculator250includes a speech score calculator251, a music score calculator252, and a noise score calculator253. The score calculator250calculates the speech score SS, the music score SM, and the noise score SM, by the speech score calculator251, the music score calculator252, and the noise score calculator253, respectively. The score calculator250calculates the speech score SS, the music score SM, and the noise score SN, based on the speech and music discrimination score S1and the music and background sound discrimination score S2. The score calculator250supplies the calculated speech score SS, the music score SM, and the noise score SN to the gain controller280.

The environmental sound analyzer260subjects the environmental sound received by the environmental sound input terminal124to FFT processing, and thereby calculates a frequency characteristic (Step S14). In addition, the environmental sound analyzer260analyzes the environmental sound (Step S15). For example, the environmental sound analyzer260compares the frequency characteristic with the audio signal, and supplies a comparison result to the gain controller280. In addition, for example, the environmental sound analyzer260regularizes the environmental sound, and supplies the regularized value as a power value Pmic to the gain controller280.

The gain controller280controls the gain in accordance with the main component of the audio signal, based on the speech score SS, the music score SM, and the noise score SN which are supplied from the score calculator250, and the comparison result supplied from the environmental sound analyzer260(Step S16). Thereby, the gain controller280determines the gain. The gain controller280supplies the gain to the equalizer220.

The equalizer220corrects the audio signal, based on the gain supplied from the gain controller280, as described above. Specifically, the equalizer220multiplies the audio signal by the gain (Step S17), and thereby obtains a corrected audio signal. The equalizer220outputs the corrected audio signal (Step S18).

Next, operation of the score calculator250will be explained hereinafter in detail.

For example, the score calculator250calculates the speech and music discrimination score S1and the music and background sound discrimination score S2, by using a linear discrimination function. However, the score calculator250may use any method other than the linear discrimination function.

The score calculator250stores in advance weight coefficients A and B to be multiplied by the various characteristic parameters which are necessary for calculating the speech and music discrimination score S1and the music and background sound discrimination score S2. The weight coefficients of a larger value are provided for a characteristic parameter which has larger effect for identifying the sound type. Specifically, the weight coefficients A and B are stored in advance for each characteristic parameter used for calculation of the scores.

The score calculator250selects a plurality of characteristic parameters (characteristic parameter set) to be used for calculation of the speech and music discrimination score S1, from the various characteristic parameters calculated by the characteristic parameter calculator240. The characteristic parameter set is represented by, for example, a vector x.

When the characteristic parameter set x includes n characteristic parameters, the speech and music discrimination score S1is calculated by “S1=A0+A1·x1+A2·x2+ . . . +An·xn”.

When the characteristic parameter set y includes m characteristic parameters, the music and background sound discrimination score S2is calculated by “S2=B0+B1·y1+B2·y2+ . . . +Bm·ym”.

The coefficient A is set such that the difference between the speech and music discrimination score S1and 1.0 is minimum when the main component of the audio signal is music, and the difference between the speech and music discrimination score S1and −1.0 is minimum when the main component of the audio signal is speech.

The coefficient B is set such that the difference between the music and background sound discrimination score S2and 1.0 is minimum when the main component of the audio signal is music, and the difference between the music and background sound discrimination score S2and −1.0 is minimum when the main component of the audio signal is background sound.

FIG. 5toFIG. 7illustrate an example of processing performed by the audio controller200.

The score calculator250calculates the speech and music discrimination score S1and the music and background sound discrimination score S2, based on the various characteristic parameters supplied from the characteristic parameter calculator240(Step S21).

The score calculator250determines whether the speech and music discrimination score S1is a negative number or not (Step S22). When it is determined at Step S22that the speech and music discrimination score S1is a negative number, the score calculator250determines whether the music and background sound discrimination score S2is a positive number or not (Step S23).

When it is determined at Step S23that the music and background sound discrimination score S2is a positive number, the speech score calculator251of the score calculator250sets the speech score SS as the absolute value of the speech and music discrimination score S1(Step S24). In addition, the music score calculator252of the score calculator250sets the music score SM to “0” (Step S25).

When it is determined at Step S23that the music and background sound discrimination score S2is not a positive number, the speech score calculator251of the score calculator250sets the speech score SS to “SS=|S1|+αs·|S2|” (Step S2.6). Specifically, the speech score calculator251adds a value obtained by correcting the absolute value of the music and background sound discrimination score S2by a preset coefficient αs to the absolute value of the speech and music discrimination score S1, to take the speech component included in the background noise included in the audio signal into consideration. In addition, the music score calculator252of the score calculator250sets the music score SM to “0” (Step S27).

The score calculator250updates correction values (stabilizing correction values) SS3and SM3to stabilize the speech score SS and the music score SM, respectively, which are calculated at Step S24to Step S27(Step S28). For example, when the speech score SS is a positive number (SS>0) successive Cs times or more, the score calculator250updates the stabilizing correction value SS3and the stabilizing correction value SM3.

In this case, the score calculator250updates the stabilizing correction value SS3to a value (SS3+βs) which is obtained by adding a preset predetermined stabilizing coefficient βs to the already calculated stabilizing correction value SS3, as a new stabilizing correction value SS3. In addition, the score calculator250updates the stabilizing correction value SM3to a value (SM3-γm) which is obtained by subtracting a preset predetermined stabilizing coefficient γm from the already calculated stabilizing correction value SM3, as a new stabilizing correction value SM3.

When it is determined at Step S22that the speech and music discrimination score S1is not a negative number, the score calculator250goes to Step S31ofFIG. 6. The score calculator250determines whether the music and background sound discrimination score S2is a positive number or not (Step S31).

When it is determined at Step S31that the music and background sound discrimination score S2is a positive number, the speech score calculator251of the score calculator250sets the speech score SS to “0” (Step S32). In addition, the music score calculator252of the score calculator250sets the music score SM as the speech and music discrimination score S1(Step S33).

When it is determined at Step S31that the music and background sound discrimination score S2is not a positive number, the speech score calculator251of the score calculator250sets the speech score SS to “SS=−S1+αs·|S2|” (Step S34). Specifically, the speech score calculator251adds a value, which is obtained by correcting the absolute value of the music and background sound discrimination score S2by a preset coefficient αs, to a negative number (−S1) of the speech and music discrimination score S1, to take the speech component included in background noise included in the audio signal into consideration.

In addition, the music score calculator252of the score calculator250sets the music score SM to “SM=S1−αm·|S2|” (Step S35). Specifically, the music score calculator252subtracts a value, which is obtained by correcting the absolute value of the music and background sound discrimination score S2by a preset coefficient αm, from the speech and music discrimination score S1, to take the music component included in background noise included in the audio signal into consideration.

The score calculator250updates the correction values (stabilizing correction values) SS3and SM3to stabilize the speech score SS and the music score SM calculated at Step S32to Step S35(Step S36). For example, when the music score SM is a positive number (SM>0) successive Cm times or more, the score calculator250updates the stabilizing correction value SS3and the stabilizing correction value SM3.

In this case, the score calculator250updates the stabilizing correction value SS3to a value (SS3-γs) which is obtained by subtracting a preset predetermined stabilizing coefficient γs from the already calculated stabilizing correction value SS3, as a new stabilizing correction value SS3. In addition, the score calculator250updates the stabilizing correction value SM3to a value (SM3+βm) which is obtained by adding a preset predetermined stabilizing coefficient βm to the already calculated stabilizing correction value SM3, as a new stabilizing correction value SM3.

In addition, the score calculator250clips the stabilizing correction value SS3and the stabilizing correction value SM3updated at Step S28ofFIG. 5and Step S36ofFIG. 6(Step S29). Thereby, the score calculator250controls the stabilizing correction value SS3and the stabilizing correction value SM3to fall within a predetermined range. For example, the score calculator250performs control such that the stabilizing correction value SS3has a value which falls within a range “SS3min≦SS3≦SS3max”. In addition, for example, the score calculator250performs control such that the stabilizing correction value SM3has a value which falls within a range “SM3min≦SM3≦SM3max”.

Then, the score calculator250goes to Step S41ofFIG. 7. The score calculator250stabilizes the speech score SS and the music score SM, based on the stabilizing correction value SS3and the stabilizing correction value SM3clipped at Step S36(Step S41). For example, the score calculator250adds the stabilizing correction value SS3to the already calculated speech score SS. The score calculator250also adds the stabilizing correction value SM3to the already calculated music score SM.

The score calculator250calculates a noise and non-noise discrimination score S3(Step S42). For example, the score calculator250calculates statistics of the characteristic parameter SFM, and thereby generates the noise and non-noise discrimination score S3. In this case, the score calculator250calculates the noise and non-noise discrimination score S3, based on the spectral flatness for each of a plurality of frequency bands (low range, middle range, high range).

The score calculator250determines whether the noise and non-noise discrimination score S3is a positive number or not (Step S43). When it is determined at Step S43that the noise and non-noise discrimination score S3is a positive number, the noise score calculator253of the score calculator250sets the noise score SN as the noise and non-noise discrimination score S3(Step S44).

When it is determined at Step S43that the noise and non-noise discrimination score S3is not a positive number, the noise score calculator253of the score calculator250sets the noise score SN to “0” (Step S45).

In addition, the score calculator250performs stabilizing correction and clipping for the noise score SN set at Step S46(Step S46).

The score calculator250performs inter-score adjustment correction (Step S47). Specifically, the score calculator250adjusts balance between the speech score SS, the music score SM, and the noise score SN. For example, when the music score SM and the noise score SN are larger than preset values, the score calculator250corrects the music score SM to reduce the music score SM in accordance with the noise score SN.

The score calculator250supplies the speech score SS, the music score SM, and the noise score SN calculated by the above method to the gain controller280.

The environmental sound analyzer260calculates a masking level, based on the environmental sound received by the environmental sound input terminal124and a preset masking characteristic. The masking characteristic is obtained by numerically expressing human ears' sensitivity for sound for each frequency. For example, the environmental sound analyzer260performs time-frequency transform on a signal of the environmental sound, and calculates a frequency characteristic of the environmental sound. The environmental sound analyzer260superposes the masking characteristic on the frequency characteristic of the environmental sound, and thereby calculates a masking level.

In addition, the environmental sound analyzer260calculates a masking gain for each frequency, based on the masking level and the audio signal. Thereby, the environmental sound analyzer260calculates a masking gain to correct the audio signal such that the audio signal is not buried in the environmental sound.

As illustrated inFIG. 8, the environmental sound analyzer260superposes the masking level on the frequency characteristic (power) of the audio signal. The environmental sound analyzer260calculates a gain (masking gain) to correct the frequency characteristic of the audio signal to a value equal to or higher than the masking level, for each frequency band. The environmental sound analyzer260supplies the calculated masking gains to the gain controller280.

Next, operation of the gain controller280will be explained in detail hereinafter.

FIG. 9illustrates an example of operation of the gain controller280.

FIG. 9illustrates an example of gain control shown by Step S16ofFIG. 4. The gain controller280receives the speech score SS, the music score SM, and the noise score SN supplied from the score calculator250. The gain controller280also receives the regularized environmental sound Pmic from the environmental sound analyzer260.

The gain controller280determines a sound type which is dominant in the audio signal. Specifically, the gain controller280determines a sound type which is dominant in the audio signal, based on the speech score SS, the music score SM, and the noise score SN supplied from the score calculator250.

The gain controller280determines whether the music score SM is highest or not (Step S51). Specifically, the gain controller280compares the speech score SS, the music score MM and the noise score SN, and determines whether the music score SM is highest or not. When it is determined that the music score SM is highest, the gain controller280determines that music is dominant. Specifically, the gain controller280determines that music is the main component of the audio signal.

When it is determined at Step S51that music is dominant, the gain controller280performs gain stabilizing processing described later (Step S52). Thereby, the gain controller280calculates a stabilized gain. When the music component is dominant, it is effective to increase the volume to change the playback sound to be heard more easily. When the sound has high followability to an environmental sound which greatly fluctuates, however, the volume of the music fluctuates, and the music is difficult to hear. Therefore, in this case, it is desirable to stabilize the gain value to be corrected.

In addition, the gain controller280corrects the gain of the whole bands to be large (Step S53). For example, the gain controller280corrects the gain such that the volume of the whole bands increases in accordance with the average intensity of the environmental sound.

The gain controller280outputs the corrected gain to the equalizer220(Step S54).

When it is determined at Step S51that music is not dominant, the gain controller280performs masking correction (Step S55). Specifically, the gain controller280sets the masking gain which is supplied from the environmental sound analyzer260, as the gain.

In addition, the gain controller280determines whether the noise score SN is less than a preset threshold level or not (Step S56). When it is determined that the noise score SN is less than the preset threshold level, the gain controller280goes to Step S53. When it is determined that the noise score SN is equal to or higher than the preset threshold level, the gain controller280goes to Step S54.

Thereby, the gain controller280can perform control such that the gain is further increased when the noise is small. In addition, when the noise is large, the gain controller280outputs the predetermined gain without any processing. As a result, the gain controller280can control the gain in accordance with intensity of the noise. The gain controller280may have a structure of amplifying the predetermined gain at a percentage smaller than that adopted at Step S53, when the noise is large. When a noise component as well as human speech is included in the playback sound itself, these noise components are amplified by increasing the volume, and thereby the sound becomes difficult to hear. Therefore, although the whole volume is increased when it is found that the noise score of the playback sound is not large, the whole volume is not increased or moderately increased when the noise score is large. Therefore, appropriate volume control can be performed according to the content of the playback sound.

FIG. 10illustrates an example of gain stabilizing processing.

The gain controller280performs gain stabilizing processing, and thereby suppresses fluctuations of the value of the gain to be multiplied by the audio signal in which music is dominant.

First, the gain controller280calculates a temporary gain Gtmp (Step S61). For example, when the power of the regularized environmental sound supplied from the environmental sound analyzer260is Pmic and a preset coefficient is α, the gain controller280calculates the temporary gain Gtmp based on the following numerical expression 1.

The gain controller280determines whether the music score SM is equal to or higher than a preset threshold level ThMsLv (Step S62). When it is determined at Step S62that the music score SM is equal to or higher than the preset threshold level, the gain controller280clips the temporary gain Gtmp (Step S63). Specifically, when the calculated temporary gain Gtmp has a value which falls out of a preset gain range, the gain controller280controls the temporary gain Gtmp to fall within the preset range.

For example, when the value of the calculated temporary gain Gtmp exceeds the preset range, the gain controller280calculates an upper limit value of the preset range as the temporary gain Gtmp. When the value of the calculated temporary gain Gtmp is less than the preset range, the gain controller280calculates a lower limit value of the preset range as the temporary gain Gtmp.

The gain controller280may have a structure of storing a gain which is calculated based on the audio signal of the previous frame, and controlling the temporary gain Gtmp such that a difference between the stored gain and the temporary gain Gtmp is less than a preset predetermined value.

The gain controller280sets a value of the clipped temporary gain Gtmp as the gain (Step S64). Specifically, when it is determined that a sound type of the main component of the audio signal is music, the gain controller280converts the temporary gain Gtmp into a value which falls within the preset range, and sets the value as the gain.

When it is determined at Step S62that the music score SM is less than the preset threshold level, the gain controller280sets the value of the temporary gain Gtmp as the gain (Step S65).

As described above, the gain controller280controls the value of the gain to fall within the predetermined range, when the main component of the audio signal is music and the level of the music is equal to or higher than a predetermined level. Thereby, the audio controller200can correct the audio signal with a stable gain. Specifically, the audio controller200can stabilize the corrected audio signal.

Consequently, it is possible to provide an audio controlling apparatus, an audio correction apparatus, and an audio correction method, which can realize sound with higher sound quality.

FIG. 11illustrates another example of the gain stabilizing processing.

The gain controller280performs the gain stabilizing processing, and thereby suppresses fluctuations of the value of the gain to be multiplied by the audio signal in which music is dominant.

First, the gain controller280calculates a temporary gain Gtmp (Step S71). For example, the gain controller280calculates the temporary gain Gtmp based on the above Expression 1, when the power of the regularized environmental sound supplied from the environmental sound analyzer260is Pmic and a preset coefficient is α.

The gain controller280determines whether the music score SM is equal to or higher than a preset threshold level ThMsLv or not (Step S72). When it is determined at Step S72that the music score SM is equal to or higher than the preset threshold level, the gain controller280determines whether a fixed gain is not preset (Step S73). The gain controller280includes a memory which stores a fixed gain. The gain controller280determines whether a fixed gain has already been set in the memory or not. When the fixed gain has already been set, the gain controller280continuously uses the fixed gain which has already been set.

When it is determined at Step S73that the fixed gain is not set, the gain controller280determines whether the environmental sound is equal to or higher than a preset predetermined value or not (Step S74). For example, the environmental sound analyzer260compares the environmental sound with the preset predetermined value, determines whether the value of the environmental sound is equal to or higher than the preset predetermined value, and supplies a comparison result to the gain controller280. The gain controller280determines whether the environmental sound is equal to or higher than the preset predetermined value, based on the result supplied from the environmental sound analyzer260.

When it is determined at Step S74that the environmental sound is equal to or higher than the preset predetermined value, the gain controller280stores the calculated temporary gain Gtmp to a predetermined memory (Step S75). The gain controller280includes a memory to store the temporary gain Gtmp for a predetermined time. The gain controller280successively stores temporary gains Gtmp in the memory.

Then, the gain controller280determines whether a preset predetermined time has passed or not (Step S76). For example, the gain controller280determines whether a predetermined time has passed since the temporary gain Gtmp is first stored in the memory. The gain controller280may have a structure of determining whether at least a predetermined number of temporary gains Gtmp are stored in the memory or not. The gain controller280may have a structure of determining whether a predetermined time has passed based on the number of frames.

When it is determined at Step S76that a preset predetermined time has passed, the gain controller280smoothes the temporary gains Gtmp which are stored in the memory, and calculates a fixed gain (Step S77).

The gain controller280sets a value of the calculated fixed gain as the gain (Step S78). For example, the gain controller280calculates an average value of a plurality of temporary gains Gtmp, and sets the calculated average value as fixed gain. The gain controller280may have a structure of calculating a median based on the temporary gains Gtmp, and setting the calculated median as fixed gain. The gain controller280may have a structure of calculating an average value as described above, and further calculating an average value by using a predetermined number of temporary gains Gtmp which are close to the calculated average value.

The gain controller280sets the value of the temporary gain Gtmp as gain, when it is determined at Step S72that the music score SM is less than the preset threshold level ThMsLv, when it is determined at Step S74that the environmental sound is less than the preset predetermined value, or when it is determined at Step S76that the preset predetermined time has not passed (Step S79).

As described above, the gain controller280uses the temporary gain Gtmp as gain, until a predetermined time has passed. In addition, the gain controller280calculates a fixed gain based on the temporary gains Gtmp which are stored during a predetermined time, and uses the fixed gain as gain. According to the above structure, the gain controller280can use a more stable gain. Specifically, the gain controller280can suppress influence of a momentary environmental sound, since the temporary gain Gtmps are calculated based on change in the environmental sound during a predetermined time since the environmental sound which is not less than the predetermined value has been detected, and a fixed gain is calculated based on the calculated temporary gains Gtmp.

Thereby, the audio controller200can correct the audio signal by using a more stable gain. Specifically, the audio controller200can stabilize the corrected audio signal more effectively. Consequently, it is possible to provide an audio controlling apparatus, an audio correction apparatus, and an audio correction method, which can realize sound of higher quality.

Functions described in the above embodiment may be constituted not only with use of hardware but also with use of software, for example, by making a computer read a program which describes the functions. Alternatively, the functions each may be constituted by appropriately selecting either software or hardware.