Patent Description:
<CIT> discloses a talkback apparatus including a talkback function for talking from an operating person of a mixer to a performer, and a backtalk function for talking from a performer to an operating person (hereinafter referred to as an engineer) of a mixer.

The talkback apparatus disclosed in <CIT> cuts voice from a microphone of the performer to a speaker of the engineer while a talkback switch is ON. In addition, the talkback apparatus disclosed in <CIT> transmits backtalk voice from the microphone of the performer to the speaker of the engineer during a period from when the "ON" of the talkback switch is released to when a preset time set by a predetermined timer reaches.

The backtalk function disclosed in <CIT> becomes usable after the talkback switch is released. Therefore, the performer cannot freely use the backtalk function.

<CIT> B discloses generating a mixing request during a live performance based on sensed mouth movement when sensed audio from the mouth is less than a threshold. In such an example method, the sensed mouth movement may be generated from a sensor mounted separately from an audio sensor of a microphone, and may be displayed at a mixing console during the live performance.

<CIT> discloses a distributed self-scaling network audio processing system which includes end nodes interconnected by packet-switched network and operating as peers on the network. Each of the end nodes supports local input processing, mixing, and output processing. The input processing includes the option of dual input channels for supporting separate front-of-house and monitor workflows. End nodes are added to the system to support specific audio processing applications, based on the number of audio sources, the number of output mixes required, and the number of locations from which users choose to interact with the system.

<CIT> discloses a mixing console in which N input channel signals from an input channel section having N input channels are selectively supplied to M mixing buses and cue buses. Mixed outputs from the mixing buses are supplied, via talkback input sections, to an output channel section having M output channels, and M output channel signals from the output channel section are patched to output ports via an output patch section. Talkback signal from a talkback supply section is supplied to the talkback input sections. When a talkback function is ON, a talkback signal is mixed with the mixed outputs from the mixing buses and then output to the output channels, during which time the mixed outputs are temporarily attenuated.

<CIT> discloses a commentator or simultaneous translator system with an audio mixing console, a command intercom and an operating unit.

In view of the foregoing, a preferred embodiment of the present invention is directed to provide an audio signal processing method, an audio signal processing system, and a storage medium storing a program that are able to use a backtalk function at a desired timing of a performer.

An audio signal processing method includes the features of claim <NUM>. Preferred embodiments are defined in corresponding dependent claims <NUM> - <NUM>. An audio signal processing system includes the features of claim <NUM>. Preferred embodiments are defined in corresponding dependent claims <NUM> - <NUM>. A non-transitory computer readable storage medium includes the features of claim <NUM>.

According to a preferred embodiment of the present invention, a backtalk function is able to be used at a desired timing of a performer.

As described above, a conventional backtalk function becomes usable after a talkback switch is released. Therefore, a performer cannot freely use the backtalk function. In addition, the conventional backtalk function uses a microphone (a normal microphone for obtaining a performance sound, a singing sound, or the like) of the performer. A signal of the microphone of the performer is outputted to a main output channel and an output channel for a monitor speaker for a performer that the performer uses. However, when the backtalk function is used, the signal of the microphone of the performer needs to be outputted to a monitor speaker for an engineer that an engineer uses. Therefore, the engineer needs to perform a complicated setup in order to achieve a backtalk function using a normal microphone.

An audio signal processing system according to a preferred embodiment of the present invention is able to use a backtalk function at a desired timing of a performer. In addition, the audio signal processing system according to a preferred embodiment of the present invention is able to use the backtalk function, without a need to perform a complicated setup.

<FIG> is a block diagram showing a configuration of an audio signal processing system <NUM> according to a preferred embodiment of the present invention. The audio signal processing system <NUM> includes a mixer <NUM>, a speaker <NUM>, a terminal <NUM>, a microphone <NUM>, and a speaker <NUM>. The speaker <NUM> is an example of a monitor speaker for a performer. The speaker <NUM> is an example of a monitor speaker for an engineer. The terminal <NUM> is an information processor (such as a personal computer, a smartphone, or a tablet PC, for example) that a performer uses.

<FIG> is a block diagram showing a configuration of the mixer <NUM>. The mixer <NUM> includes a display <NUM>, an operator <NUM>, an audio I/O (Input/Output) <NUM>, a signal processor <NUM>, a PC I/O <NUM>, a MIDI I/O <NUM>, a diverse (Other) I/O <NUM>, a network I/F <NUM>, a CPU <NUM>, a flash memory <NUM>, and a RAM <NUM>.

The display <NUM>, the operator <NUM>, the audio I/O <NUM>, the signal processor <NUM>, the PC I/O <NUM>, the MIDI I/O <NUM>, the Other I/O <NUM>, the CPU <NUM>, the flash memory <NUM>, and the RAM <NUM> are connected to each other through a bus <NUM>. In addition, the audio I/O <NUM> and the signal processor <NUM> are also connected to a waveform bus <NUM> for transmitting an audio signal. It is to be noted that, as will be described below, an audio signal may be sent and received through the network I/F <NUM>. In such a case, the signal processor <NUM> and the network I/F <NUM> are connected through a not-shown dedicated bus.

The audio I/O <NUM> is an interface for receiving an input of an audio signal to be processed in the signal processor <NUM>. The audio I/O <NUM> includes an analog input port, a digital input port, or the like that receives the input of an audio signal. In addition, the audio I/O <NUM> is an interface for outputting an audio signal that has been processed in the signal processor <NUM>. The audio I/O <NUM> includes an analog output port, a digital output port, or the like that outputs the audio signal.

Each of the PC I/O <NUM>, the MIDI I/O <NUM>, and the Other I/O <NUM> is an interface that is connected to various types of external devices and performs an input and output operation. The PC I/O <NUM> is connected to an information processor such as a personal computer, for example. The MIDI I/O <NUM> is connected to a MIDI compatible device such as a physical controller or an electronic musical instrument, for example. The Other I/O <NUM> is connected to a display, for example. Alternatively, the Other I/O <NUM> is connected to a UI (User Interface) device such as a mouse or a keyboard. Any standards such as Ethernet (registered trademark) or a USB (Universal Serial Bus) are able to be employed for communication with the external devices. The mode of connection may be wired or wireless.

The network I/F <NUM> communicates with a different apparatus such as the terminal <NUM> through a network. In addition, the network I/F <NUM> receives an audio signal from the different apparatus through the network and inputs a received audio signal to the signal processor <NUM>. Further, the network I/F <NUM> receives the audio signal on which the signal processing has been performed in the signal processor <NUM>, and sends the audio signal to the different apparatus through the network.

The CPU <NUM> is a controller that controls the operation of the mixer <NUM>. The CPU <NUM> reads out a predetermined program stored in the flash memory <NUM> being a storage medium to the RAM <NUM> and performs various types of operations. It is to be noted that the program does not need to be stored in the flash memory <NUM> in the own apparatus. For example, the program may be downloaded each time from another apparatus such as a server and may be read out to the RAM <NUM>.

The display <NUM> displays various types of information according to the control of the CPU <NUM>. The display <NUM> includes an LCD or a light emitting diode (LED), for example.

The operator <NUM> receives an operation with respect to the mixer <NUM> from an engineer. The operator <NUM> includes various types of keys, buttons, rotary encoders, sliders, and the like. In addition, the operator <NUM> may include a touch panel laminated on the LCD being the display <NUM>.

The signal processor <NUM> includes a plurality of DSPs (Digital Signal Processors) for performing various types of signal processing such as mixing processing or effect processing. The signal processor <NUM> performs signal processing such as mixing processing or effect processing on an audio signal to be supplied from the audio I/O <NUM> through the waveform bus <NUM>. The signal processor <NUM> outputs a digital audio signal on which the signal processing has been performed, to the audio I/O <NUM> again through the waveform bus <NUM>.

<FIG> is a block diagram showing a function of signal processing to be performed in the signal processor <NUM>, the audio I/O <NUM>, and the CPU <NUM>. As shown in <FIG>, the signal processing is functionally performed through an input patch <NUM>, an input channel <NUM>, a bus <NUM>, an output channel <NUM>, and an output patch <NUM>.

The input patch <NUM> receives an input of an audio signal from a plurality of input ports (an analog input port or a digital input port, for example) in the audio I/O <NUM> and assigns any one of a plurality of ports to at least one of a plurality of channels (<NUM> channels, for example). As a result, the audio signal is supplied to each channel in the input channel <NUM>.

<FIG> is a diagram showing a functional configuration of the input channel <NUM> and the bus <NUM>. The input channel <NUM> includes a plurality of signal processing blocks, for example, in order from a signal processing block <NUM> of a first input channel, and a signal processing block <NUM> of a second input channel, to a signal processing block <NUM> of a 32nd input channel. Each signal processing block performs various types of signal processing such as an equalizing or compressing, to the audio signal supplied from the input patch <NUM>.

The bus <NUM> includes a stereo bus <NUM>, a MIX bus <NUM>, and a monitor bus <NUM>. Each signal processing block of the first input channel to the 32nd input channel inputs the audio signal on which the signal processing has been performed, to the stereo bus <NUM>, the MIX bus <NUM>, and the monitor bus <NUM>. Each signal processing block of the first input channel to the 32nd input channel sets an outgoing level with respect to each bus.

The stereo bus <NUM> corresponds to a stereo channel used as a main output in the output channel <NUM>. The MIX bus <NUM>, for example, corresponds to a monitor speaker (a speaker <NUM>, for example) of a performer. In other words, the MIX bus <NUM> is an example of a monitor bus for a performer. The monitor bus <NUM> corresponds to a monitor speaker (a speaker <NUM>, for example) for an engineer. Each of the stereo bus <NUM>, the MIX bus <NUM>, and the monitor bus <NUM> mixes inputted audio signals. Each of the stereo bus <NUM>, the MIX bus <NUM>, and the monitor bus <NUM> outputs the mixed audio signals to the output channel <NUM>.

The output channel <NUM>, as with the input channel <NUM>, performs various types of signal processing on the audio signal inputted from the bus <NUM>. The output channel <NUM> outputs the audio signal on which the signal processing has been performed, to the output patch <NUM>. The output patch <NUM> assigns each output channel to at least one of a plurality of ports serving as an analog output port or a digital output port. As a result, the output patch <NUM> supplies the audio signal on which the signal processing has been performed, to the audio I/O <NUM>.

An engineer sets a parameter of the above-described various types of signal processing, through the operator <NUM>. <FIG> is a diagram showing a configuration of an operation panel of the mixer <NUM>. As shown in <FIG>, the mixer <NUM> includes, on the operation panel, a touch screen <NUM> and a channel strip <NUM>. Such components correspond to the display <NUM> and the operator <NUM> shown in <FIG>. It is to be noted that, although <FIG> only shows the touch screen <NUM> and the channel strip <NUM>, a large number of knobs, switches, or the like may be provided in practice.

The touch screen <NUM> is the display <NUM> obtained by stacking the touch panel being one preferred embodiment of the operator <NUM>, and constitutes a GUI (Graphical User Interface) for receiving an operation from a user.

The channel strip <NUM> is an area in which a plurality of physical controllers that receive an operation with respect to one channel are disposed vertically. Although <FIG> only shows one fader and one knob for each channel as the physical controllers, a large number of knobs, switches, or the like may be provided in practice. In the channel strip <NUM>, a plurality of faders and knobs disposed on the left side correspond to the input channel <NUM>. The two faders and two knobs disposed on the right side are physical controllers corresponding to the master output. An engineer operates a fader and a knob, sets a gain of each input channel (from the first input channel to the 32nd input channel in a case of <NUM> channels, for example), or sets an outgoing level with respect to the bus <NUM>.

<FIG> is a block diagram showing a configuration of the terminal <NUM>. The terminal <NUM> includes a display <NUM>, a microphone <NUM>, a flash memory <NUM>, a RAM <NUM>, a network I/F <NUM>, a CPU <NUM>, and a touch panel <NUM>. The display <NUM>, the microphone <NUM>, the flash memory <NUM>, the RAM <NUM>, the network I/F <NUM>, the CPU <NUM>, and the touch panel <NUM> are connected to each other through a bus <NUM>.

The network I/F <NUM> communicates with another apparatus such as the mixer <NUM> through a network. The network I/F <NUM> sends various types of information to the mixer <NUM> through the network.

The CPU <NUM> reads out a program stored in the flash memory <NUM> being a storage medium to the RAM <NUM> and implements various types of functions. It is to be noted that the program that the CPU <NUM> reads out does not also need to be stored in the flash memory <NUM> in the own apparatus. For example, the program may be stored in a storage medium of an external apparatus such as a server. In such a case, the CPU <NUM> may read out the program each time from the server to the RAM <NUM> and may execute the program.

<FIG> shows an example of a screen displayed on the display <NUM>. It is to be noted that the display <NUM> includes the touch panel <NUM> stacked on the display <NUM>, and constitutes a GUI together with the touch panel <NUM>. The touch panel is an example of a receptor. The display <NUM> displays a setting screen for performing a volume setting as shown in <FIG>. A performer, on the setting screen, sets the volume of sound according to an audio signal to be sent out to the speaker <NUM> being a monitor speaker for a performer.

On the setting screen, names of each performance sound or each singing sound corresponding to an input channel are arranged. The vertical direction corresponds to volume (gain). In this example, a drum (DRUMS), a base (BASS), a vocal (VOCAL), and a guitar (GUITAR) are displayed along in the horizontal direction. A bar that moves in the vertical direction is displayed at a position at which each performance sound or each singing sound is displayed.

A performer, by touching the display <NUM> on which the touch panel <NUM> is stacked and performing a swipe operation to move the bar up and down, can change the outgoing level of the input channel corresponding to each name. For example, when the bar of the DRUMS is moved upward, from the input channel corresponding to the DRUMS, the outgoing level with respect to the MIX bus <NUM> corresponding to the monitor speaker of the performer is able to be increased.

The CPU <NUM> of the terminal <NUM> sends information according to the volume setting received by the touch panel <NUM>, to the mixer <NUM>. The information according to the volume setting includes information to specify the MIX bus <NUM> and information on the outgoing level. The mixer <NUM> receives the information according to the volume setting from the terminal <NUM> and sets an outgoing level with respect to the corresponding MIX bus <NUM>.

Subsequently, when the performer touches a backtalk button <NUM> in the setting screen, the backtalk function is enabled. When the backtalk function is enabled, the performer can talk with an engineer using the microphone <NUM> of the terminal <NUM>.

<FIG> is a flow chart showing an operation of the audio signal processing system. First, the CPU <NUM> of the terminal <NUM> determines whether or not a backtalk input instruction has been received (S11). For example, when a performer touches the backtalk button <NUM> in the setting screen, the CPU <NUM> receives a backtalk input instruction. The CPU <NUM> repeats the determination of S11 until receiving a backtalk input instruction.

The CPU <NUM>, when receiving a backtalk input instruction, obtains an audio signal obtained by the microphone <NUM> as a backtalk signal (S12). Then, the CPU <NUM> sends the backtalk input instruction and the backtalk signal through the network I/F <NUM> (S13).

The CPU <NUM> of the mixer <NUM> receives the backtalk input instruction and the backtalk signal through the network I/F <NUM> (S21). The CPU <NUM> sets the signal processor <NUM> so as to send out the backtalk signal to the monitor bus <NUM> (S22). For example, the signal processor <NUM>, in the input channel <NUM> shown in <FIG>, sets the backtalk switch <NUM> to an ON state. The signal processor <NUM>, when setting the backtalk switch <NUM> to the ON state, comes into the state of sending out the backtalk signal received from the terminal <NUM> to the monitor bus <NUM>. As a result, the signal processor <NUM> outputs the backtalk signal obtained by the microphone <NUM> to the speaker <NUM> being a monitor speaker for an engineer. Therefore, the engineer can listen to the voice of a performer.

In such a manner, an audio signal processing system according to a preferred embodiment of the present invention is able to use a backtalk function at a desired timing of a performer. In addition, the audio signal processing system according to a preferred embodiment of the present invention is able to use the backtalk function, without a need to perform a complicated setup.

In addition, when the backtalk function is enabled, the signal processor <NUM> reduces the level of an audio signal other than a backtalk signal (S23). As shown in <FIG>, signals of the signal processing block <NUM> to the signal processing block <NUM> are also inputted to the monitor bus <NUM>. Therefore, in a state in which the backtalk function is not enabled, the engineer is in a state of listening to sound of any input channel. In such a state, even when the backtalk signal is sent out to the monitor bus <NUM>, the engineer can listen to the voice of a performer. In such a case, when the signal processor <NUM> reduces the level of an audio signal other than a backtalk signal, the engineer can clearly listen to the voice of a performer. However, the processing of S23 is not essential in the present invention.

It is to be noted that the mixer <NUM> is connected to the microphone <NUM>. The microphone <NUM> is a talkback microphone for talking from an engineer to a performer. An engineer operates the operator <NUM> of the mixer <NUM>, and turns on a talkback function. The mixer <NUM> receives a talkback input instruction, for example through a talkback button provided on the operation panel. In such a case, the signal processor <NUM> outputs a talkback signal according to voice obtained by the microphone <NUM> to a monitor bus for a performer other than the monitor bus <NUM>. In the example of <FIG>, the signal processor <NUM> turns on a talkback switch <NUM> in the input channel <NUM>. The signal processor <NUM>, when turning on the talkback switch <NUM>, sends out the talkback signal received from the microphone <NUM>, to a bus other than the monitor bus <NUM>. As a result, the signal processor <NUM> outputs the talkback signal obtained by the microphone <NUM> to the speaker <NUM> being a monitor speaker for a performer. Therefore, the performer can listen to the voice of an engineer.

In addition, when the talkback function is enabled, the signal processor <NUM> may reduce the level of an audio signal other than a talkback signal. When the signal processor <NUM> reduces the level of an audio signal other than a talkback signal, the performer can clearly listen to the voice of an engineer.

It is to be noted that backtalk voice is not limited to an example of outputting from the speaker <NUM>. For example, the mixer <NUM> may connect to a terminal (a second terminal) that an engineer uses, and may output the backtalk voice from the speaker of the terminal that the engineer uses.

Next, <FIG> is a diagram showing a functional configuration of an input channel <NUM> and a bus <NUM> according to a modification. The signal processor <NUM> according to the modification, when turning on the backtalk switch <NUM>, sends out the backtalk signal received from the terminal <NUM> not only to the monitor bus <NUM> but to a specified bus. Therefore, the voice of the performer when the backtalk function is turned on is outputted not only from a monitor speaker of an engineer but also from any specified monitor speaker of a performer.

<FIG> is a flow chart showing an operation of an audio signal processing system according to the invention. The same reference numerals are used to refer to processing common to the processing shown in <FIG>, and the description is omitted. The CPU <NUM> of the terminal <NUM>, in a case of determining that a backtalk input instruction has been received in S11, further receives a specification of an outgoing destination of the backtalk signal (S102). A performer, by selecting a name of each performance sound or each singing sound on the setting screen shown in <FIG>, for example, can specify the outgoing destination of a backtalk signal.

The CPU <NUM> of the terminal <NUM> sends a backtalk input instruction, a backtalk signal, and information on the outgoing destination, through the network I/F <NUM> (S103).

The CPU <NUM> of the mixer <NUM> receives the backtalk input instruction, the backtalk signal, and the information on the outgoing destination, through the network I/F <NUM> (S201). The CPU <NUM>, based on the information on the outgoing destination, sets the signal processor <NUM> so as to send out the backtalk signal to a bus of the specified outgoing destination (S202). For example, the signal processor <NUM>, in the input channel <NUM> shown in <FIG>, sets the backtalk switch <NUM> to an ON state. The signal processor <NUM>, when turning on the backtalk switch <NUM>, comes into the state of sending out the backtalk signal received from the terminal <NUM> to the stereo bus <NUM>, the MIX bus <NUM>, and the monitor bus <NUM>. Further, the signal processor <NUM>, based on the information of the outgoing destination, sets a high outgoing level with respect to the specified bus of the outgoing destination and sets a low outgoing level with respect to other buses (S23). As a result, the signal processor <NUM> outputs the backtalk signal obtained by the microphone <NUM> to any specified speaker. Therefore, the performer can talk with any specified performer or engineer.

It is to be noted that, even in the modification, the backtalk voice is not limited to an example of outputting from the speaker <NUM>. For example, the mixer <NUM> may connect to a terminal that each performer uses, and may output the backtalk voice from the speaker of the terminal that the performer uses.

The description of the foregoing preferred embodiments is illustrative in all points and should not be construed to limit the present invention. The scope of the present invention is defined not by the foregoing preferred embodiment but by the following claims. Further, the scope of the present invention is intended to include all modifications within the scopes of the claims.

Claim 1:
An audio signal processing method comprising:
receiving, by a terminal (<NUM>), a backtalk input instruction from a performer;
obtaining, by a microphone (<NUM>) connected to the terminal (<NUM>), first voice information from the performer;
outputting, in a case where the backtalk input instruction has been received by the terminal (<NUM>), a backtalk signal corresponding to the first voice information obtained by the microphone (<NUM>) connected to the terminal (<NUM>) to a monitor bus (<NUM>) of a mixer (<NUM>); and
outputting a talkback signal corresponding to second voice information from an engineer obtained by a talkback microphone (<NUM>) to a bus comprised in the mixer (<NUM>) different from the monitor bus (<NUM>),
wherein the talkback microphone (<NUM>) is connected to the mixer (<NUM>),
characterized in that the audio signal processing method further comprises: receiving, by the terminal (<NUM>), specification of an outgoing destination of the backtalk signal; and outputting, by the mixer (<NUM>), the backtalk signal to the specified outgoing destination,
wherein the specification of an outgoing destination of the backtalk signal is done by the performer.