Switcher control device, switcher control method, and image synthesizing apparatus

A switcher control device includes: a superimposition determining unit that determines whether a superimposition image is superimposed on an image taken by a camera based on angle information and zoom value information about the camera in selecting a camera image when an effects switcher selects the image taken by the camera as a background image; and a switcher instructing unit that instructs the effects switcher to superimpose the superimposition image based on a determined result of the superimposition determining unit.

CROSS-REFERENCE TO RELATED APPLICATION

The present application claims priority from Japanese Patent Application No. JP 2008-288719 filed in the Japanese Patent Office on Nov. 11, 2008, the entire content of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a switcher control device, a switcher control method, and an image synthesizing apparatus. More specifically, the invention relates to a switcher control device in which when an effects switcher selects an image taken by a camera as a background image, the switcher control device determines whether a superimposition image is superimposed on the image taken by the camera based on angle information and zoom value information about the camera, and instructs the effects switcher to do superimposition, thereby economically implementing superimposition according to shooting states without causing degradation of the overall functions.

2. Description of the Related Art

In a relayed broadcast such as a sports broadcast, in order to represent descriptions of a stadium and players, or comments made by commentators, graphics such as subtitles and lines are sometimes superimposed in the picture area of the relayed broadcast. Superimposition of subtitles is a classic technique having been conducted. However, the positions of players are changing every moment, and it is necessary to make graphics drawings for commentary as matched with the image at that moment.

A CG (Computer Graphics) device is used for creating such superimposition images, and the process of superimposing computer graphics on a relayed image is often conducted by a studio device called an effects switcher. In the effects switcher, a circuit that handles superimposing images is called a keyer. In the effects switcher, a plurality of keyers is generally provided.

Heretofore, for operating live broadcasting, a large number of staffs is manned such as camera operators, CG device operators, effects switcher operators, audio mixer operators, image material editors, and an administrator to switch news. In recent years, such a technique has been developed that a single computer system controls a variety of types of devices for labor savings.

JP-A-H04-157882 (Patent Document 1) describes a technique that when the pan, tilt and zoom values of a camera fall in a predetermined condition, character information and graphics information stored in advance are superimposed on an image taken by the camera.

SUMMARY OF THE INVENTION

In a broadcasting system, it becomes common that processes such as selection and superimposition of images and special effects are processed by a device such as an effects switcher in a concentrated manner. For example, it can be considered that in application of the technique described in Patent Document 1 in which character information and graphics information are superimposed on the image taken by the camera, a portion of the circuit in the effects switcher is exclusively used. In this case, even though any images taken by the camera are not used, it is difficult to allocate that portion of the circuit for other purposes, which causes degradation of the overall functions.

For example, it can be considered that in application of the technique described in Patent Document 1 in which character information and graphics information are superimposed on the image taken by the camera, it can be considered that a superimposition device for cameras is provided separately from the effects switcher. In this case, although the use of the image taken by the camera is involved in only a small portion of operations, this takes a lot of money.

Thus, it is desirable to economically implement superimposition according to the shooting state of a camera without causing degradation of the overall functions in relation to a system using an effects switcher. In addition, it is also desirable to automate adjusting of the position at which an image is superimposed, to downsize the staff necessary to operate a broadcasting system to curtail costs, and to reduce the possibilities of occurrence of operating error caused by manual operations of superimposed positions. In addition, it is also desirable to economically implement suited superimposition in operations of a plurality of cameras.

An embodiment of the invention is directed to an image synthesizing apparatus including: a camera; an effects switcher that synthesizes an image; and a control unit that controls an operation of the effects switcher, wherein when the effects switcher selects an image taken by the camera as a background image, the control unit determines whether a superimposition image is superimposed on the image taken by the camera based on angle information and zoom value information about the camera, and instructs the effects switcher to superimpose the superimposition image based on the determined result.

In the embodiment of the invention, the effects switcher synthesizes an image. The operation of the effects switcher is controlled by the control unit. The image taken by the camera is selected by the effects switcher as a background image, for example.

When the effects switcher selects the image taken by the camera as a background image, the control unit determines whether a superimposition image is superimposed on the image taken by the camera based on angle information (directions of pan and tilt) and zoom value information about the camera. For example, when a target is included in the image taken by the camera, it is determined that a superimposition image is to be superimposed. Here, for example, when a target is a person, a superimposition image is the person's name.

For determining whether a target is included in the image taken by the camera, target position information and camera position information are used, for example, in addition to angle information and zoom value information about the camera. In this case, when the camera is a movable camera, information about the moved position of the camera is used.

In the control unit (switcher control device), the effects switcher is instructed to superimpose a superimposition image based on the determined result. For example, when a plurality of cameras exists, the control unit holds information about a superimposition image that is to be superimposed on each of the images taken by the individual cameras. In this case, when the effects switcher selects an image taken by a predetermined camera among a plurality of the cameras, the effects switcher is instructed to superimpose a superimposition image that is to be placed on the image taken by the predetermined camera.

The control unit controls the operation of the effects switcher based on a broadcast programming delivery program (cue sheet=playlist), for example. Then, the broadcast programming delivery program stores therein an instruction in which when the effects switcher selects the image taken by the camera as a background image, the effects switcher is instructed to superimpose the superimposition image based on information about a shooting state of the camera (such as the position, the angle, and the zoom value).

As described above, in the embodiment of the invention, when the effects switcher selects an image taken by a camera as a background image, the control unit automatically determines whether a superimposition image is to be superimposed, and instructs the effects switcher to do superimposition based on the determined result. In this case, in order to superimpose the superimposition image on the image taken by the camera, a portion of the circuit in the effects switcher is not exclusively used, the portion of the circuit is used as necessary. In addition, in this case, a superimposition device for cameras is not provided separately from the effects switcher. Therefore, superimposition according to the shooting state of the camera can be economically implemented without causing degradation of the overall functions.

In addition, in the embodiment of the invention, when the effects switcher selects an image taken by a predetermined camera among a plurality of the cameras, it is determined whether a superimposition image is superimposed on the image taken by the camera based on information about the shooting state of the predetermined camera (such as the position, the angle, and the zoom value). Then, the effects switcher is instructed to superimpose a superimposition image that is to be placed on the image taken by the predetermined camera based on the determined result. Therefore, even though a plurality of the cameras is operating, suited superimposition can be economically implemented.

In the embodiment of the invention, for example, the control unit may further calculate a position at which a superimposition image is to be superimposed based on angle information and zoom value information about the camera, and based on the calculated result, the control unit may instruct the effects switcher of the position at which the superimposition image is to be superimposed. In this case, because adjusting the superimposed position of the image is automated, the staff necessary to operate a broadcasting system can be downsized to curtail costs, and the possibilities of occurrence of operating error caused by manual operations of superimposed positions can be reduced.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, the best mode for implementing an embodiment of the invention (hereinafter, referred to as an “embodiment”) will be described. In addition, the descriptions will be provided in the order below.

1-1. Exemplary Configuration of a Broadcast Programming Delivery Control System

FIG. 1shows an exemplary configuration of a broadcast programming delivery control system100as an embodiment. This broadcast programming delivery control system100has an automation control block10, a news room control system (NRCS) block20, an MOS device block30, and an automation control device block40.

The control block10controls broadcast programming delivery based on a playlist (cue sheet) formed of a plurality of configurations. Here, the playlist configures a broadcast programming delivery program, and individual cues (individual formations) of the playlist correspond to individual items of the broadcast programming delivery program. The NRCS block20conducts management and administration related to news programs such as arrangements for gathering information, placing orders of materials, and making formations of broadcast programming. In the MOS (Media Object Server) device block30, a playout server31, a CG/still store32, and other units are arranged, which are networked to the NRCS block20(according to MOS Protocol). In the control device block40, a switcher41, a device control unit (DCU)42, a video clip server43, and an audio clip server44are arranged, which are connected to the control block10. In addition, in the control device block40, a video tape recorder (VTR)45, a robotics camera (Robotics Camera)46, and an audio mixer47are arranged.

The NRCS block20is connected to the control block and to the MOS device block30through an NRCS network according to the MOS protocol. The MOS protocol is a protocol for remotely controlling a video system server installed in a news room from the NRCS, and allowed to have original extended tags based on XML (Extensible Markup Language).

On an NRCS client, an automation plug-in (NRCS Plug-in) and a playlist viewer are installed. To the individual MOS devices in the MOS device block30, the device control unit42in the automation control device block40issues an on-air timing by triggers such as Tally/GPI.

Here, the playout server31is a video server that stores materials changed and replaced everyday, which is demanded to have advanced functions because the playout server is used for editing. In addition, the playout server31has playlists and a MOS interface. It is necessary that the playout server31quickly conducts feeding and editing materials, and thus a server of higher performance than that of the video clip server43is often used for the playout server31. In addition, depending on types of the playout server31, because it is difficult to continuously use the same channel for outputting different materials, channel assignment control in consideration of this point is necessary.

1-2. Exemplary Configuration of Software Blocks of the Broadcast Programming Delivery Control System

FIG. 2shows an exemplary configuration of software blocks of the broadcast programming delivery control system100. In other words, the broadcast programming delivery control system100has a database (DB) unit101, an NRCS plug-in/playlist viewer unit102, and an NRCS interface (NRCS IF) unit103. In addition, the broadcast programming delivery control system100has an on-air playlist/setup unit104, a device controller unit105, and a log/alarm manager unit106. In the broadcast programming delivery control system100, the individual units construct an automation system.

Automation is TDA (Technical Director Assist) software on a computer unit, which controls devices including the switcher under the NRCS. On the other hand, a server in the automation system also has playlists. Generally, a producer (P) produces an overall news program, and a technical director (TD) conducts the assignment and setting of devices in individual stories in the program. It is the automation system that assists the technical director to control devices. The playlist is a list that holds the detail of progress of a news program in a time series.

In the automation system, playlist data is created by the NRCS of the NRCS plug-in/playlist viewer unit102(1), and through a MOS gateway (MOS GW)103A of an NRCS interface unit103, the playlist data is sent to the database unit101(2). Then, the playlist data is stored in a database (DB) of the database unit101.

The playlist data is displayed on a playlist viewer (Playlist Viewer)102A of the NRCS plug-in/playlist viewer unit102, and sent to a device control unit105when a program is on the air (3). The devices are controlled to go on the air in accordance with the playlist (4), and the device statuses are notified to the database (DB) of the database unit101(5) and (6). Then, the playlist and the device statuses are sent to the on-air playlist/setup unit104, and displayed (7). A log or warning is generated in the automation system, and sent to the log/alarm manager unit106.

1-3. Exemplary Implementation of the Software Blocks of the Broadcast Programming Delivery Control System

The individual software blocks in the automation system can be split and installed into individual personal computers for each software block, and a scalable system can be offered as matched with customer's system resizes.

FIG. 3shows an exemplary implementation of the software blocks of the broadcast programming delivery control system100. In this exemplary implementation, the database unit101, the NRCS interface unit103, the on-air playlist/setup unit104, the device control unit105, and the log/alarm manager unit106are installed in a single computer unit110. In addition, the NRCS plug-in/playlist viewer unit102is installed in a customer's computer unit120in which an NRCS client is installed.

FIG. 4shows another exemplary implementation of the software blocks of the broadcast programming delivery control system100. In this exemplary implementation, the database unit101is installed in a single computer unit110A. In addition, the log/alarm manager unit106is installed in a different computer unit110E. Moreover, the NRCS interface unit103, the on-air playlist/setup unit104, and the device control unit105are installed in a different computer unit110B. In addition, the NRCS plug-in/playlist viewer unit102is installed in the customer's computer unit120in which the NRCS client is installed.

FIG. 5shows still another exemplary implementation of the software blocks of the broadcast programming delivery control system100. In this exemplary implementation, the database unit101is installed in a single computer unit110A. In addition, the log/alarm manager unit106is installed in a different computer unit110E. Moreover, the NRCS interface unit103, the on-air playlist/setup unit104, and the device control unit105are installed in separate computer units110B,110C, and110D. In addition, the NRCS plug-in/playlist viewer unit102is installed in the customer's computer unit120in which the NRCS client is installed.

As discussed above, the log/alarm manager unit106is installed in the standalone computer unit110E, whereby independence can be provided to detection of abnormal systems. In addition, the database unit101is installed in the standalone computer unit110E, whereby the degree of freedom can be provided to the database system configuration.

1-4. Detailed Block Configuration of the Database Server and Units Around It

Here, the detailed block configuration of the database server (DB Server) of the database unit101and units around it in the automation system will be described.FIG. 6shows an exemplary block configuration of the database server and units around it.

The NRCS interface unit103includes the MOS gateway (MOS GW)103A, an object editor (Object Editor)103B, an object manager (Object Manager)103C, and a MOS device interface (MOS Device I/F)103D.

The MOS gateway103A communicates with the NRCS block20according to the MOS protocol, and updates the database unit101. In other words, the MOS gateway103A receives a RUNDOWN (Running Order) from the NRCS block, and converts the RUNDOWN into a playlist and writes it in the database unit101.

The object editor103B makes setting to objects of devices connected to the automation system. Here, the devices connected to the automation system are the switcher41, the audio mixer47, the robotics camera46, the video clip server43, and other units. The object editor103B is activated from the NRCS block20, and displays, creates, and changes objects on the database. In addition, the object editor103B exchanges objects with the database unit101through the object manager103C.

The object manager103C manages objects edited by the object editor103B, reads objects out of the database unit101, and writes objects in the database unit101. The MOS device interface103D acquires information such as the status of the MOS device and channel assignments, and writes the information in the database unit101.

In addition, the NRCS plug-in/playlist viewer unit102graphically displays a created playlist, or the state of a playlist on the air. The NRCS plug-in/playlist viewer unit102includes the playlist viewer102A that displays an automation playlist and an on-air playlist (OA Playlist)102B that displays a playlist on the air.

The playlist viewer102A displays the detail of a playlist created and changed in the NRCS block20. The on-air playlist102B displays and manages the progress of a playlist going on the air. The on-air playlist102B displays a playlist on the air, and displays the current on-air position. In addition, the on-air playlist102B displays the standby situations of video and audio materials. Moreover, the on-air playlist102B specifies subsequent items/formations.

The device control unit105controls various devices based on a playlist through consoles (CCP for MVS, and JL Cooper Modules). The device control unit105includes an on-air manager (OA Manager)105A, an event controller105B, a device controller105C, and a manual controller (Manual Controller)105D.

The on-air manager105A manages on air broadcasting. In other words, the on-air manager105A notifies activation to the device control unit105by manipulations of activating and finishing a playlist, and conducts standby and finishing processes of a playlist.

The event controller105B executes a single event in a playlist. In other words, the event controller105B instructs the device controller105C about operations necessary at the next standby time, based on a playlist on the database unit101.

In addition, the event controller105B instructs a series of operations conducted when on air (TAKE). In other words, the event controller105B sends an on-air timing (Take Trigger) to the device controller105C. In this case, the event controller105B sends an on-air timing to the device controller105C by creating and sending a symbolic command/time line.

In addition, the event controller105B stores device statuses in the database unit101.

The manual controller105D generates an on-air timing (Take Trigger) manually, by notification from a master (Master Switcher), or according to time. In addition, the manual controller105D conducts assignable event/device control by a utility/shot box module. In addition, the manual controller105D conducts control necessary for playlist execution such as TAKE manipulations and dialogues. In addition, the manual controller105D executes temporary events by quick recall. Moreover, the manual controller105D controls devices independently.

The device controller105C controls devices. In other words, the device controller105C interprets an abstracted symbolic command, and transforms it into a predetermined protocol to control devices. In addition, the device controller105C transforms the protocol into an abstracted status, and notifies it to the event controller105B. In addition, the device controller105C holds the time line of the symbolic command, and controls synchronization by an on-air timing (Take Trigger).

The on-air playlist/setup unit104makes settings of the automation core system and configurations of control devices by a setup/configuration104A. The log/alarm manager unit106collects various logs and alarms scattered to individual terminals by a log/alarm manager106A, and when a trouble occurs, the log/alarm manager unit106notifies it to an SNMP manager. The database unit101collectively manages information such as playlists, events, settings, and statuses.

1-5. Configurations of Essential Units of the Broadcast Programming Delivery Control System100, and Configurations of Essential Software Modules

FIG. 7shows an exemplary configuration of essential units of the broadcast programming delivery control system100. In addition,FIG. 8shows an exemplary configuration of essential software modules in the broadcast programming delivery control system100.

In other words, the broadcast programming delivery control system100has the automation control block10and the NRCS block20. The automation control block10controls programmed broadcast programming delivery based on the software blocks installed in the first computer unit (Automation Ctrl PC)110. The NRCS block20conducts management and administration related to a news program such as arrangements for gathering information, placing orders of materials, and making formations of broadcast programming, based on the software blocks installed in the second computer unit120.

In addition, the broadcast programming delivery control system100has the MOS device block30formed of MOS devices such as the playout server31, and the CG/still store32. The MOS device block30is networked to the second computer unit120.

In addition, the broadcast programming delivery control system100has the automation control device block40formed of the switcher41, the device controller (DCU)42, the video clip server43, the audio clip server44, the VTR45, the robotics camera46, and the audio mixer47. The automation control device block40is networked to the first computer unit110.

In addition, the broadcast programming delivery control system100has a control unit (SCU)130having a video reference input. In addition, the broadcast programming delivery control system100has a communication channel140connecting between the control unit130and the first computer unit110, and a network150networking the first computer unit110to the second computer unit120.

On the control unit130, an event executer105E is mounted, which is an execution and management module sending a video signal reference (Ref. In) to the first computer unit110through the communication channel140.

On the first computer unit110, a software module that manages playlist information and a device driver software module in conformity to video devices to be controlled are mounted. In addition, on the first computer unit110, an event execution module is mounted. The event execution module has a software interface capable of adding a device driver software module, and a software interface to a device control module in the control unit130.

The event executor105E is a software module operated by a microcomputer of the control unit130. Ref. In is a reference input, and reference signals (for example, black burst signals) of the entire equipment are supplied to the control unit130. The control unit130is configured to interrupt the built-in microcomputer by the reference signals at one-field intervals.

When this interrupt occurs, the event executor105E notifies the other software module on the same microcomputer, for example, a manual event controller105F about that. At the same time, the event executor105E also notifies the first computer unit (automation Ctrl PC)110through the communication channel140. More specifically, the event executor105E is in a wait state for tasks. The event executor105E receives a wait cancelation from an interrupt process routine by the reference signal to again start to run a task for the above-described process, and then enters the wait state again when finishing it.

The manual event controller105F receives an interrupt notification to organize several processes at one-field intervals. For example, the control unit130has a push button to control the switcher. The manual event controller105F sends the event that the push button has been pressed as a command to the switcher41that is the automation control device in the automation control device block40through the communication channel.

This transmission is conducted at one-field intervals, whereby no delay occurs in control, and degraded efficiency can be avoided because of very frequent communications. The manual event controller105F accumulates information about the pressed button, forms information in a command format as one field of press-down information by interrupt notification, and then sends it to the switcher41. In addition, the manual event controller105F sends the command to the switcher41as well as sends information about the pressed button to the first computer unit (automation Ctrl PC)110.

The event controller105B that is a software module in the first computer unit (automation Ctrl PC)110controls execution of control events for various devices in accordance with playlist information. The progress of the playlist is conducted according to time code. The event controller105B receives notification from the control unit130at one-field intervals, whereby the event controller105B can control the progress with the use of this notification as clock information. When the system is operating according to interlacing, because two fields make one frame, receiving two notification increases time code by one.

When time code reaches a predetermined position (time position) in the playlist, the operation at this position is executed. Here, suppose that the device controller that is the software module on the first computer unit110is considered to be a first device controller, and the device controller that is the software module on the control unit130is considered to be a second device controller. The event controller105B sends an instruction in accordance with the playlist to at least any one of the first device controller and the second device controller.

Both of the device controllers transform the instruction into a native protocol of a device to be controlled (control protocol of each device), and sends it from individual ports (RS422, and Ethernet (registered trademark)).

A device controlled by an interface that a PC does not generally has, such as RS422 or a dedicated LAN, can be connected to DCU42 for control, DCU42 covering the port unit of the control unit130. On the other hand, a device connected to a multipurpose Ethernet can be connected to the PC for similar control.

The device controller (second device controller) on the control unit130synchronizes with reference signals for control, in order to control the switcher (Switcher)41and the video tape recorder45in the automation control device block40. Consequently, in the broadcast programming delivery control system100, a device controlled from the first computer unit (automation Ctrl PC)110side can be synchronized with the control unit130side for operation.

It is unnecessary to provide hardware to receive reference signals in the first computer unit (automation Ctrl PC)110, and the communication channel according to Ethernet is used, whereby costs can be suppressed. In addition, flexible configurations as shown inFIGS. 3,4, and5are feasible, and multipurpose products can be used in replacing the computer unit, which is economical.

The control unit130conducts transformation into a native protocol of the device connected to the control unit130. On this account, loads are not concentrated on the first computer unit (automation Ctrl PC)110side, and loads are distributed to economically execute device control with no delay in synchronization with reference signals.

InFIG. 8, the master connected to the event executor105E is a master switcher. The master switcher is a switching apparatus at higher level than the switcher and this system in a broadcast station, and the master switcher allocates video signals in the overall broadcast station, while this system belongs to a specific studio.

The control unit (SCU)130receives instructions and timings from the master (Master) through a port such as a GPI (General Purpose Interface). The GPI is the port to receive instructions and triggers in response to reception of simple ons/offs in a parallel port. The master switches outputs from the station to studio outputs or CM. In switching from a CM to a studio output, the automation receives a trigger from the GPI to start the formations after the CM. In this case, the master takes the initiative of switching. Conversely, such an operation may be possible that a frame called “CM” is provided in a formation, a trigger is given to the master from the studio by a GPI output in the reverse direction, and then the master switches to a CM.

In addition, the event executor105E relays among the master switcher, the other software module, and the first computer unit (automation Ctrl PC)110. It is necessary to assure that delay in relay does not exceed a defined field unit. To this end, when the operation is started at the timing according to the reference signal, a GPI input is read. When the level is changed, it is determined what instruction is made in accordance with the setting of the GPI port, and a notification is made to the other software module and/or to the first computer unit (automation Ctrl PC)110correspondingly.

When an output is made from the GPI, the event executor105E processes the instruction received from the other software module and the first computer unit (automation Ctrl PC)110in units of fields. In other words, when the operation is started at the timing according to the reference signal, the event executor105E organizes the received instructions to change the level of GPI output in accordance with the settings of the individual GPI ports.

The event executor105E has a plurality of processes to conduct at the timing according to the reference signals. Executing every field is essential to reduce the delay in this execution to one field or below (two fields or below from reception to completion of transmission). The upper limit of delay can be ensured even though the order during the process is not strictly defined.

InFIG. 6, a Microsoft SQL Server can be used as a data server of the database unit101. When a certain application rewrites the database unit101, the data server uses an MS SQL function to notify the other application of a changed item. This is called a dispatcher. Properly speaking, the individual applications resister items desired to notify, receive change notification, and then reads the database unit101. When alteration is made in a formation after the current point in time in a playlist, or the order of formations is changed, the first computer unit110uses this function to prepare change in control.

As described above, in the broadcast programming delivery control system100, the control unit130having the reference input sends a timing for reference to the first computer unit110. Then, the event executor105E receives this timing for reference, and controls the device driver software module in the first computer unit110and the device control module in the control unit130.

Thus, in the broadcast programming delivery control system100, video devices having various control operations interfaces can be controlled in synchronization with the timing for reference in accordance with the playlist, and delivery control can be implemented at accurate timings also in conjunction with the NRCS block20.

1-6. Configuration of the Switcher

Next, the switcher41will be described.FIG. 9shows an exemplary configuration of the switcher41. The switcher41has a main unit440, and a select input unit460connected to the main unit440through a communication channel450. The main unit440has a matrix switcher unit410, a first image processing unit420A, a second image processing unit420B, and a control unit430. The first image processing unit420A and the matrix switcher unit410corresponding to this configure a first ME bank. In addition, the second image processing unit420B and the matrix switcher unit410corresponding to this configure a second ME bank.

The matrix switcher unit410selects video signals inputted to input lines L1to L12. To the first image processing unit420A and the second image processing unit420B, video signals inputted to the input lines L1to L12are supplied through the matrix switcher unit410. The control unit430controls the operations of the matrix switcher unit410, the first image processing unit420A, and the second image processing unit420B.

To the input lines L1to L12of the switcher41, video signals to be key source signals or key fill signals and video signals to be background video signals are inputted. The matrix switcher unit410has key source cross-points412A1and412A2. The key source cross-points412A1and412A2are formed of switches connected to key source signal select input buses411A1and411A2that supply one of video signals inputted to the input lines L1to L12as a key source signal to the first image processing unit420A.

In addition, the matrix switcher unit410has key fill cross-points414A1and414A2. The key fill cross-points414A1and414A2are formed of switches connected to key fill signal select input buses413A1and413A2that supply one of video signals inputted to the input lines L1to L12as a key fill signal to the first image processing unit420A.

In addition, the matrix switcher unit410has first background cross-points416A. The first background cross-points416A are formed of switches connected to a first background signal select input bus415A that supplies one of video signals inputted to the input lines L1to L12as a first background signal to the first image processing unit420A. In addition, the matrix switcher unit410has second background cross-points418A. The second background cross-points418A are formed of switches connected to a second background signal select input bus417A that supplies one of video signals inputted to the input lines L1to L12as a second background signal to the first image processing unit420A.

In addition, the matrix switcher unit410has key source cross-points412B1and412B2. The key source cross-points412B1and412B2are formed of switches connected to key source signal select input buses411B1and411B2that supply one of video signals of the input lines L1to L12and output lines of the first ME bank as a key source signal to the second image processing unit420B.

In addition, the matrix switcher unit410has key fill cross-points414B1and414B2. The key fill cross-points414B1and414B2are formed of switches connected to key fill signal select input buses413B1and413B2that supply one of video signals of the input lines L1to L12and output lines of the first ME bank as a key fill signal to the second image processing unit420B.

In addition, the matrix switcher unit410has first background cross-points416B. The first background cross-points416B are formed of switches connected to a first background signal select input bus415B that supplies one of video signals inputted to the input lines L1to L12and output lines of the first ME bank as a first background signal to the second image processing unit420B. In addition, the matrix switcher unit410has second background cross-points418B. The second background cross-points418B are formed of switches connected to a second background signal select input bus417B that supplies one of video signals inputted to the input lines L1to L12and output lines of the first ME bank as a second background signal to the second image processing unit420B.

The first image processing unit420A is formed of key processing circuits421A1and421A2and a combination circuit422A. The key processing circuits421A1and421A2are connected to the key source signal select input buses411A1and411A2and the key fill signal select input buses413A1and413A2. To the key processing circuits421A1and421A2, a key source signal and a key fill signal selected from video signals inputted to the input lines L1to L12are inputted through the key source signal select input buses411A1and411A2and the key fill signal select input buses413A1and413A2. The key processing circuits421A1and421A2generate a key signal corresponding to a control signal from the control unit430by the inputted key source signal or by an incorporated waveform generation circuit (wipe pattern generation circuit). Then, the key processing circuits421A1and421A2supply a key signal and a key fill signal to the combination circuit422A.

The combination circuit422A is connected to the key processing circuits421A1and421A2as well as connected to the first background signal select input bus415A and the second background signal select input bus417A. To the combination circuit422A, the key signal and the key fill signal are inputted from the key processing circuits421A1and421A2. In addition, to the combination circuit422A, a first background signal and a second background signal selected from the video signals inputted to the input lines L1to L12are inputted through the first background signal select input bus415A and the second background signal select input bus417A. The combination circuit422A selectively uses the key signal and the key fill signal supplied from the key processing circuit421A1or the key processing circuit421A1under control conducted by the control unit430. Then, the combination circuit422A conducts a keying process in which the area indicated by the key signal is replaced by the first background signal or the second background signal to combine a key fill signal under control conducted by the control unit430.

The second image processing unit420B is formed of key processing circuits421B1and421B2and a combination circuit422B. The key processing circuits421B1and421B2are connected to the key source signal select input buses411B1and411B2, and the key fill signal select input buses413B1and413B2. To the key processing circuits421B1and421B2, a key source signal and a key fill signal selected from video signals of the input lines L1to L12and output lines of the first ME bank are inputted through the buses411B1and411B2and the key fill signal select input buses413B1and413B2. The key processing circuits421B1and421B2generate a key signal corresponding to a control signal from the control unit430by the inputted key source signal or by an incorporated waveform generation circuit (wipe pattern generation circuit). Then, the key processing circuits421B1and421B2supply a key signal and a key fill signal to the combination circuit422B.

The combination circuit422B is connected to the key processing circuits421B1and421B2as well as connected to the first background signal select input bus415B and the second background signal select input bus417B. To the combination circuit422B, a key signal and a key fill signal are inputted from the key processing circuits421B1and421B2. To the combination circuit422B, a first background signal and a second background signal selected from video signals of the input lines L1to L12and output lines of the first ME bank are inputted through the first background signal select input bus415B and the second background signal select input bus417B. The combination circuit422B selectively uses a key signal and a key fill signal supplied from the key processing circuit421B1or the key processing circuit421B1under control conducted by the control unit430. Then, the combination circuit422B conducts a keying process in which the area indicated by the key signal is replaced by the first background signal or the second background signal to combine a key fill signal under control conducted by the control unit430.

In the descriptions above, the description is made in which the key signal is binary signals and the background signal or the key fill signal is specified. However, actually, the key signal is not binary signals, and the concentration of laying the key fill signal on the background signal is shown by multiple values. Therefore, such image processing can be also conducted in which a portion having the key fill signal semitransparently seen is provided while the background is viewed.

The control unit430is configured of a microcomputer, for example. The control unit430generates a control signal corresponding to a select input signal given from the select input unit460thorough the communication channel450, and controls the individual operations of the matrix switcher unit410, the first image processing unit420A, and the second image processing unit420B through control lines435A and435B.

The select input unit460is formed of a microcomputer465to which a button arrangement unit461, a keyboard462, a pointing device463such as a mouse, and a graphical display464are connected. When the buttons of the button arrangement unit461are manipulated to select input, in the matrix switcher unit410of the main unit440, the cross-points connected to the corresponding input bus are controlled to select input, and a video signal is supplied to the corresponding image processing unit. Then, video signals are processed such as image synthesis specified in the image processing unit, and video signals to form a picture area including the video signal of the selected input are outputted.

In addition, in the case of dual link mode in which video signals of progressive mode are processed in pairs, image paths are provided in pairs to be doubled on A side and B side and operations are made such that paired inputs are selected on each of A side and B side.

In the switcher41shown inFIG. 9, the select input unit460communicates with the control unit430of the main unit440through the communication channel450to instruct execution of various processes. The combination circuit422A receives inputs of the background signals from the first background signal select input bus415A and the second background signal select input bus417A, and uses any one of the background signals for the keying process to form a background signal in accordance with the instruction from the select input unit460. Alternatively, the combination circuit422A combines two background signals at the instructed ratio to form a background signal used for the keying process in accordance with the instruction from the select input unit460.

The ratio is manually instructed by a fader lever of the select input unit, or in the case of operations of automatic transition (automatic progress), the ratio is changed over time, and controlled such that one of the background signals is switched to the other background signal. As one example of the combination method, for example, there is a method in which two background signals are weighted and added for each pixel at the above-described ratio. For instance, suppose that the ratio is 30%, the value that the value of the first background signal is multiplied by 0.3 is added to the value that the value of the second background signal is multiplied by 0.7.

In addition, as another example of the combination method, there is a method in which a second background signal is superimposed on a first background signal with a wipe key signal supplied from a wipe key generator (WKG) by keying. The key signal generated in the wipe key generator is changed with the above-described ratio, and the key signal is generated such that the time of day of the progress is used as an input parameter to determine the boundary line of wipe in the case of automatic transition. In the case in which it is not automatic transition, the instructed ratio is used to generate the key signal instead of time of day. The descriptions above are the same in the case of the combination circuit422B.

FIG. 10shows a diagram depicting the appearance of the console400of the switcher41. The console400is a manipulating unit corresponding to one of the image processing units of the main unit440when the mode is not dual link mode. When the mode is dual link mode, the console400is a manipulating unit corresponding to the paired image processing units.

As shown inFIG. 10, the console400is provided with first background buttons401, second background buttons402, and key buttons403-1and403-2. In addition, the console400is provided with source name indicators404, transition target buttons405A and405B, direction specifying buttons406A and406B, a round-trip mode specifying button407, and a fader lever408.

The first background buttons401are manipulated when the first background signal select input bus415A selects the input line through which the video signals to be background video signals are supplied to the combination circuit422A. The second background buttons402are manipulated when the second background signal select input bus417A selects the input line through which the video signals to be background video signals are supplied to the combination circuit422A.

The key buttons403-1and403-2are manipulated when they select the combination of the first input line and the second line through which the video signals to be key source signals and key fill signals are supplied to the key processing circuits421A1and421A2. In this case, the key source signal select input buses411A1and411A2supply the video signals to be key source signals to the key processing circuits421A1and421A2through the first input line. In addition, the key fill signal select input buses413A1and413A2supply the video signals to be key fill signals to the key processing circuits421A1and421A2through the second input line.

InFIG. 10, the source name indicators404displays text information associated with the index number corresponding to the button number of the button arranged below. Text information is stored in the control unit430, and a user can set text information. In addition, text information may be stored as associated with the input line number. In this case, any one item of text information associated with the first input line number corresponding to the index number corresponding to the button number of the button and text information associated with the second input line number corresponding to the index number is displayed.

The first input line number is a number to select the video signal to be the key fill signal, and the second input line number is a number to select the video signal to be the key source signal. Thus, a pair of the first input line number and the second input line number is associated with the index number corresponding to the button number. Therefore, the key buttons can select the combination of the first and second input lines through which the video signals to be the key source signal and the key fill signal are supplied to the key processing circuit.

When the background buttons are manipulated, the background video signal is selected by the first input line number. Such a scheme may be possible that the source name indicator404generally displays text information associated with the first input line number, and displays text information associated with the second input line number only while a separately provided button, not shown, is pressed down. Thus, the operator can confirm text information of the first and second line numbers. In addition, associations of the index number corresponding to the button number and the input line number corresponding to the index number are called cross-points button assignments. On the console400, the selected button (switch) is lit.

In the broadcast programming delivery control system100, the NRCS block20obtains channel assignment information acquired from the playout server31, and automatically selects the cross-points (XPT) of the switcher41set in the playlist in the automation control block10. In other words, according to the MOS protocol, channel assignment information in the playout server31is passed to the NRCS block20.

The automation control block10obtains channel assignment information in the playout server31by means of roStorySend according to the MOS protocol or FTP. Then, the automation control block10sets and changes the cross-points of the switcher41based on channel assignment information.

In creating a playlist, when video sources are selected, the setting is made as only the playout server31, and no channel is set. In other words, on the display screen of the graphical display464, when SVR is selected on a palette464A shown inFIG. 11, SVR is set on a playlist464B shown inFIG. 12. However, no channel is set. When the automation control block10obtains channel information from the NRCS block20, the automation control block10displays channel information in the playout server31on the playlist to set the cross-points of the switcher41.

In other words, in the broadcast programming delivery control system100, the automation control block10installed in the first computer unit110executes the playlist. In this case, the automation control block10references to information about the material passed from the NRCS block20, channel assignment information and the playlist, when the playout server31enters the state of outputting a certain material from the output channel.

Then, the automation control block10conducts a process of instructing a proper bus of the switcher41to select an input connected to the output channel of the playout server31in accordance with the material select schedule in the playlist and the process schedule of the switcher41. For example, when a certain material is used as background video in the playlist, for the proper bus, any one of the background signal select input buses is fit, and then an instruction is made to select the corresponding input in the cross-points.

Alternatively, when a material for subtitles is keyed, the corresponding input is selected at the cross-points of the key source signal select input bus and the key fill signal select input bus, and then the material is used. Alternatively, when a sub-picture area is provided according to picture in picture, the corresponding input is selected by the key fill signal select input bus to obtain video signals of a material. Then, the video signals are subjected to compression, key signals only for the sub-picture area generated inside the key processing circuit, and the sub-picture area is formed for keying without using video signals from an external unit through the key source signal select input bus.

Then, on the display part of the source name indicators404on the console400of the switcher41at the position corresponding to an operator to select an input connected to the output channel of the playout server31, information about the material obtained from the first computer unit110is displayed. Here, the operator to select the input connected to the output channel of the playout server31is the first background buttons401, the second background buttons402, and the key buttons403-1and403-2.

In the control unit430, the switcher41has a correspondence storage unit capable of setting correspondences between the operators of the console400and the corresponding inputs of the switcher41. The first computer unit110sends an instruction about the above-described correspondence for the input of the switcher41scheduled for use in the program through the output channel of the playout server31, and assigns the correspondence to the operator of the console400. In addition, the instruction about the correspondence includes material information, and this material information is displayed on the source name indicator404as corresponding to the operator of the console400in this example.

In addition, other than the relation to the playout server31, material servers such as the video clip server43and the audio clip server44may similarly change the correspondence also in changing a material outputted from a certain output channel.

1-7. Specific Example of Broadcast Programming Delivery

Next, a specific example of broadcast programming delivery in the broadcast programming delivery control system100will be described. Here, suppose that the output channels of the material server are CH1and CH2. Then, materials outputted from the output channels CH1and CH2are supposed to be clip A, clip B, and clip C. The output channels CH1and CH2of the material server are supposed to be connected to the inputs1and2of the switcher41. In addition, in the playlist (cue sheet), materials are used in the order shown in Table 1 in image processing conducted by the switcher41. In this case, an example is shown that the progress of the playlist follows time of day.

TABLE 1Timet1t2t3t4t5BackgroundABABAP in P————BSubtitle——CC—

In other words, although the number of channels for use at the same time is two channels, three types of materials are used. Although the delivery control computer executes the above-described playlist, materials are outputted in the order shown in Table 2 because the number of the output channels of the material server is two. This detail is output channel information for the entire time of day.

Because the control of the switcher41is instructed as input1and input2, the order is shown in Table 3.

Consequently, to the switcher41, instructions are made in the order shown in Table 4. In addition, in Table 4, the numbers indicate the input numbers.

TABLE 4Timet1t2t3t4t5Background11221P in P————2Subtitle——11—

For automatic control, when cross-points control (control of the matrix switcher unit) is executed as discussed above, scheduled pictures can be outputted. Suppose that the possibility that operations are switched to manual operations is taken into account, preferably, material titles (A, B, C) are displayed on source name displays corresponding to the cross-points buttons of the console400of the switcher41. In order to implement this, source name information is supplied to the switcher (the system including the console), the associations with material titles are executed. The source name information gives the switcher41a title (character string) corresponding to each input.

Table 5 shows inputs of the switcher41and exemplary sources (signal sources).

TABLE 5Switcher input numberSource1:Ch1 of material server 12:Ch2 of material server 13:Ch1 of material server 24:Ch2 of material server 25:Ch3 of material server 26:VTR 17:VTR 28:Studio camera 19:Studio camera 210:Outside broadcast van 111:Outside broadcast van 212:Outside broadcast van 3

In addition, Tables 6 and 7 show exemplary settings of cross-points button assignments. Here, a single input line number is simply assigned to a button number.

For example, in assignments in Table 6, because input line number1is assigned to button number1, for instance, “CH1of the material server1” is displayed on the indicator corresponding to button number1in accordance with Table 5. In addition, for example, “outside broadcast van1” is displayed on the indicator corresponding to button number5.

As described above, instructions are made from the automation control block10installed in the first computer unit110such that only signal sources for use in a broadcast programming are assigned to buttons. Depending on types of the playout server31, it is difficult to continuously use the same channel for outputting different materials. Thus, channel assignment control in consideration of this point is necessary.

In this case, for example, even though a program (broadcast programming) uses a single material all the time, two channels (CH1, CH2) are used for operations. For example, in the playout server31, control is conducted in such a way that material A is outputted from CH1, material B is then outputted from CH2, material C is outputted from CH1, and material A is outputted from CH2.

To the cross-points of the switcher41, inputs connected to CH1and CH2are assigned. Then, in using materials in the progress of the broadcast programming, cross-points are controlled so as to select the corresponding input. Thus, in the broadcast programming, a material used at each point in time is specified, it is unnecessary for operator to be aware from which channel that material is outputted, and the material is correctly selected.

As described above, it is sufficient that the first computer unit110acquires from the playout server31which material is outputted from which channel with enough lead time to prepare control before the timing of each switching. This acquisition may be made before broadcast programming is started, and it is sufficient that the above-described lead time is kept even though the broadcast programming is ongoing.

Because the automation control block10knows materials used in the playlist, the automation control block can identify materials for use from the playlist. In addition, because materials for use are assigned to buttons, even though operations are suddenly moved to manual operations, necessary materials can be selected.

In addition, on the console400of the switcher41, cross-points buttons are generally capable of selecting doubled inputs with the use of a “shift” button. However, because manipulations with pressing the shift button cause time and effort, it is convenient that materials for frequent use are allocated to buttons of smaller numbers, not with the shift button. Even though buttons are provided enough, when inputs for use in the broadcast programming are assigned to buttons of smaller numbers, the ease of manipulation is improved in manual operations.

In addition, even though the automation control block10makes cross-points button assignments, because the material title is displayed on the source name indicators404on the console400of the switcher41, manual manipulations including VTR1and VTR2have no troubles.

1-8. Superimposition on an Image Taken by a Camera

Next, superimposition on an image taken by a camera will be described.FIG. 13schematically shows components related to superimposition on an image taken by the camera46in the broadcast programming delivery control system100shown inFIG. 1described above. InFIG. 13, components corresponding to those inFIGS. 1 and 7are designated the same numerals and signs for properly omitting the detailed descriptions.

The controller110controls multiple cameras, three cameras46and a single effects switcher41in the example shown. The controller110stores a playlist (cue sheet) therein, and conducts broadcast programming delivery control according to the playlist, that is, the controller110controls the operations of the camera46, the effects switcher41, and other devices. The shooting state of the camera46is remotely controlled by the controller110, or manually operated by an operator operating the camera46. Here, the shooting state includes the position of the camera46, and the angle (directions of pan and tilt) and zoom value of the camera46.

The video signals acquired by taking images by a plurality of the cameras46are connected to the inputs of the effects switcher41. The effects switcher41also receives other video signals to its inputs, selects an image, and applies a special effect to that image to create an output image (On Air). A manipulation console41A controls the effects switcher41by manual operations. A system control console111is used to manipulate the controller110. An operator uses the system control console111to create a playlist, to make instructions, and to conduct manual operations for other devices to be controlled.

In the embodiment, the controller110(first computer unit (Automation Ctrl)) determines whether a superimposition image is superimposed on an image taken by the camera when the effects switcher41selects the image taken by the camera (remote controlled camera)46as a background image. Then, the controller110instructs the effects switcher41to superimpose the superimposition image based on the determined result.

When a target (target subject) such as a person is included in an image taken by the camera46, for example, the controller110determines that a superimposition image is to be superimposed. Here, the superimposition image is subtitles to explain a target. For example, when a target is a person, the superimposition image is the target person's name. The controller110calculates the shooting area of the camera46based on angle information and zoom information about the camera46, and considers that the target is included in the image taken by the camera46when the position at which the target exists is included in the shooting area.

In addition, when the effects switcher41selects an image taken by the camera (remote controlled camera)46as a background image, the controller110calculates the position at which a superimposition image is to be superimposed based on angle information and zoom value information about the selected camera46. Then, the controller110instructs the effects switcher41of the position at which the superimposition image is to be superimposed based on the calculated result. The details of the calculating methods of the superimposed position and the shooting area described above will be described later.

FIGS. 14A to 14Care diagrams illustrative of superimposition. For the purpose of generating an image, an intention is to desire that a person name be displayed for a person at a specific position in the area of a taken image and the name be positioned below that person.FIG. 14Cshows an exemplary superimposition image. The video signals of the superimposition image are externally supplied to the input of the effects switcher41(from the CG/still store32, the playout server (Playout Server)31, and others), or read out of a memory incorporated in the effects switcher41.

For example, when the video signals of the superimposition image are acquired from the material server, superimposition images are stored according to identifiers of material clips, not according to input numbers. In operations, which input the effects switcher41selects for a predetermined keyer is determined based on the correspondence between the material clip (identifier) sent from the material server and the output channel of the material server.

FIG. 14Ashows an exemplary image that a first camera (camera1) takes a person and a superimposition image is positioned below the person for superimposition.FIG. 14Bshows an exemplary image that a second camera (camera2) takes a person and a superimposition image is positioned below the person in the picture area for superimposition. In the embodiment, when the effects switcher41selects an image taken by the camera46as a background image, the image shown inFIG. 14AorFIG. 14Bis automatically created as an output image.

FIG. 15is a diagram illustrative of operating conditions of a plurality of the cameras46. A camera46-1as the camera1and a camera46-2as the camera2shoot from different positions. In each of the cameras, control of the shooting state, that is, control of the position, the angle (directions of pan and tilt), and the zoom value is independently conducted. In this case, depending on shooting states, a target is sometimes included in the images taken by the individual cameras, or not. In the example shown, a target (target person) is included both in the images taken by the camera46-1and the camera46-2.

1-9. Calculating Method of the Shooting Area

FIG. 16is a diagram depicting an exemplary calculating method of the shooting area. Suppose that the position coordinates of the camera46are (x0,y0), and the subject is positioned on the line y=0. In addition, when the camera46is a fixed camera, the coordinates (x0,y0) are the fixed values. On the other hand, when the camera46is a movable camera, the coordinates (x0,y0) express the moved position of the camera46. The controller110calculates the values of x1 and x2 indicating the shooting area on the line y□0 from Equation (1) and Equation (2) below.
x1=(y0×tan(R−T/2))+x0  (1)
x2=(y0×tan(R+T/2))+x0  (2)
where the angle of the camera46is R as shown in the drawing and the angle of the shooting area determined by the zoom value is T.

The values x1 and x2 indicate the shooting area in the horizontal direction. Although the detailed descriptions are omitted, the shooting area in the vertical direction is similarly calculated.

When the position of a target, the position of a target person, for example, is included in the calculated shooting areas both in the horizontal direction and the vertical direction as described above, the controller110determines that the target is included in the taken image. In addition, in this case, the target may be treated as a point. However, it may be determined whether the target is included in the shooting area in consideration of the size of the target (width in the horizontal direction and width in the vertical direction) as necessary.

FIG. 17is a diagram illustrative of a zoom value. Suppose that the shooting area of the angle of the camera46is T. The shooting area W of the subject at the distance L is expressed by Equation (3).
W□2×L×tan(T/2)  (3)

When the zoom is expressed by the value, because it is easier to tell the zoom by the expression of the apparent magnification, the zoom is sometimes expressed by W, that is, the ratio of tan(T/2). However, the transformation into the angle T can be easily performed by Equation (3) described above.

1-10. Calculating Method of the Superimposed Position

In order to find the position at which the superimposition image is to be superimposed, the controller110first calculates the target position (target subject) in the picture area. The calculating method of the position in the picture area will be described with reference toFIG. 18. In addition, inFIG. 18, the portions corresponding to those inFIG. 16are designated the same numerals and signs.

When the target position is at the position (xt,0), the relation of Equation (4) is held, where the angle of the incident light between the center of the taken image ((x1+x2)/2,0) and the target is V (clockwise).
xt=(y0×tan(R+V))+x0  (4)

This relation is used to obtain the angle V for the target at the position (xt,0). Here, as shown inFIG. 18, the value of the angle V is a positive value when the target position (xt,0) is on the right side with respect to the center of the taken image ((x1.x2)/2,0). In addition, the value of the angle V is a negative value when the target position (xt,0) is on the left side with respect to the center of the taken image ((x1.x2)/2,0).

When the angle of the incident light at the end of the picture area is (T/2), the distance from the camera to the target is generally long enough for the shooting area, and the ratio between the length from the center of the picture area to the end and the length from the center of the picture area to the target is expressed by Equation (5). The controller110can obtain the target position at the position (xt,0) in the picture area from Equation (5).
tan(V)/tan(T/2)  (5)

In addition, the descriptions above show the calculating method of the target position in the horizontal direction in the picture area. Although the detailed descriptions are omitted, the target position in the vertical direction in the picture area can be similarly calculated.

As described above, after calculating the target position in the picture area, the controller110finds the position at which a superimposition image is to be superimposed. In this case, the controller110finds the position at which a superimposition image is to be superimposed such that the superimposition image is positioned at the position related to the target position in the picture area.

For example, when the superimposition image shown inFIG. 14Cis handled, the position at which the superimposition image is to be superimposed (travel) is calculated such that the center position at the upper end of the superimposition image falls at the center position of the lower end of the target. However, when the target is positioned at the end of the picture area, the superimposition image is partially missing as the position at which the superimposition image is to be superimposed falls at the calculated position. In this case, for example, as shown inFIG. 14B, the superimposed position calculated is adjusted such that the superimposition image is included in the picture area.

For example, in the case of the superimposition image shown inFIG. 14C, the image is missing when the position is moved to the left. Thus, when the travel in the horizontal direction is negative, the travel in the horizontal direction is adjusted to zero.

1-11. Communication Sequence of the Control System

Here, an exemplary communication sequence of the control system will be described when the superimposition image is to be superimposed on the image taken by the camera46.FIG. 19shows an exemplary communication sequence of the control system. The controller110controls the shooting state of each of the cameras (a). In other words, the controller110sends instructions of positions, the angles (directions of pan and tilt), and zoom values to the cameras46-1and46-2. In addition, when the positions of the cameras46-1and46-2are fixed, the instructions of positions are omitted. In addition, the controller110sends an instruction to the effects switcher41to select an image taken by a certain camera, for example (b).

After finishing changing the position, the angle, and the zoom value, the cameras46-1and46-2send the current values of them to the controller110(c). In addition, for another example, the cameras46-1and46-2may not send any values, and the controller110may assume that the positions, the angles, and the zoom values of the cameras46-1and46-2are changed to the instructed values. In addition, for still another example, error may be returned to the controller110only when the cameras46-1and46-2do not follow the instructions from the controller110.

After receiving values such as the positions, the angles, and the zoom values from the cameras46-1and46-2, the controller110conducts determination of superimposition and calculation of the superimposed position (d). For determination of superimposition and calculation of the superimposed position, the controller110is configured to use the images from the cameras46-1and46-2as the background images in the effects switcher41, and the controller110conducts determination of superimposition and calculation of the superimposed position when a setting is made in which the corresponding superimposition image is handled. The controller110determines whether the target is included in the image taken by the camera based on the position, the angle, and the zoom value from the camera corresponding to the background image. When the target is included in the image taken by the camera, the controller110determines that a superimposition image is to be superimposed. In addition, when the controller110determines that the superimposition image is to be superimposed, the controller110also calculates the position at which the superimposition image is to be superimposed.

After conducting determination of superimposition and calculation of the superimposed position, the controller110sends instructions of superimposition and the superimposed position to the effects switcher41(e).

1-12. Functional Blocks of the Controller

As described above, the controller110controls superimposition on the image taken by the camera46.FIG. 20shows an exemplary configuration of the functional blocks related to superimposition control conducted by the controller110. The controller110has a camera control unit151, a state information receiving unit152, a superimposition determining unit153, a superimposed position calculating unit154, and a superimposition and superimposed position instructing unit155.

The camera control unit151sends to the camera46instructions of the position, the angle (directions of pan and tilt), and the zoom value, and controls the shooting state of the camera46. In addition, when the position of the camera46is fixed, the instruction of the position is not sent. In addition, when the shooting state of the camera46is manually operated by the operator at the camera46, not remotely controlled from the controller110, the camera control unit151is unnecessary.

The state information receiving unit152receives information about the shooting state such as the position, the angle, and the zoom value sent from the camera46. In addition, in such a system in which the camera46does not send information about the shooting state such as the position, the angle, and the zoom value, the state information receiving unit152is unnecessary. In this case, the superimposition determining unit153and the superimposed position calculating unit154, described later, acquire information about the shooting state such as the position, the angle, and the zoom value from the camera control unit151, and then use the information.

The superimposition determining unit153calculates the shooting area of the camera46based on information about the shooting state such as the position, the angle, and the zoom value and information about the target position received in the state information receiving unit152, and determines whether the superimposition image is to be superimposed on the image taken by the camera46based on the calculated result. In this case, when the target (target subject) is positioned in the shooting area of the camera46, the superimposition determining unit153determines to perform superimposition.

The superimposed position calculating unit154calculates the target position (target subject) in the picture area based on information about the shooting state such as the position, the angle, and the zoom value and information about the target position received in the state information receiving unit152, and finds the position at which the superimposition image is to be superimposed based on the calculated result. The superimposition and superimposed position instructing unit155instructs the effects switcher41to perform superimposition based on the determined result of the superimposition determining unit153and the calculated result of the superimposed position calculating unit154.

1-13. Exemplary Details of the Playlist Including Settings of Superimposition on the Target Position

As described above, the controller110stores the playlist (cue sheet), and controls broadcast programming delivery, that is, the operations of the camera46, the effects switcher41, and other devices according to the playlist. In the playlist, an instruction is stored in which the effects switcher41is instructed to superimpose the superimposition image when the effects switcher41selects an image taken by the camera46as a background image based on information about the shooting state of the camera46(such as the position, the angle, and the zoom value).

FIG. 21shows exemplary details of the playlist. The playlist includes the settings that the superimposition image is to be superimposed on the target position in the picture area. The playlist is stored in the controller110. In the playlist, a single row shows one formation of an output image. A formation (row) is set to automatically go to the subsequent formation (row) at a predetermined time of day (elapsed time from the start), or when a Take button is manually pressed, and any one of these settings is set in the controller110.

In the details of the individual formations shown inFIG. 21, the control state of one ME bank in the effects switcher41is shown as an example. The individual items will be described. Bkgd A is an input video signal selected for a bus background A (the bus that generally handles images outputted as background images). InFIG. 21, although the items are named as input video signals, it is the same to store the items according to numbers. Bkgd B is an input video signal selected for a bus background B.

Key1is an input video signal selected for a keyer1. Generally, for the keyer, signals are selected for two buses, a fill bus and a source bus. In this case, typically, control is configured in such a way that a number is assigned to a pair of options, the fill and the source, and this number is used to allow single number assignments. Thus, a single number is enough here.

Key1Control is the details of specifying video signal processing of the keyer1. This includes whether to lay the key (On), types of key signal processing (such as a luminous key and a chromakey), control values (such as the gain value), scaling up and down by digital image processing, and specification of modification. Key2and Key2Control are details of a keyer2as similar to the keyer1.

Transition describes a changing method in changing from the previous formation to the formation. “Cut” means a cut from the previous picture, and the picture is instantaneously changed to the subsequent picture. “Wipe3, Frames” means that a wipe of a pattern number3is used to change the picture for 30 frames. In addition to these, “Mix” means an effect that the picture is gradually synthesized for every pixel for switching, and a stored CG (computer graphics) is used to change the picture while the computer graphics is laid on the picture area.

In addition to those described above, the values related to the control state of the effects switcher41are included in one formation in the playlist.

A specification “position information1” is sometimes included in the details of Key1Control (Key2Control). This means whether to lay the key on and position control of the superimposition image depend on position information sent from the specified camera. Position information1includes information (number) to specify a camera. The camera to be specified is generally the camera selected by the bus Bkgd A. In other words, superimposition is determined from information sent from the camera shooting the background image.

In the formation including “position information1”, the effects switcher41is controlled as below. When the background image does not include the specified camera, the controller110does not conduct superimposition. On the other hand, when the background image includes the specified camera, the controller110determines whether the target (target subject) at the specific position (coordinates) is included in the shooting area (image frame) of the camera based on information about the shooting state (such as the position, the angle, and the zoom value) of this specified camera. When the target is included, the controller110further calculates the superimposed position, and instructs the effects switcher to move the superimposition image to the superimposed position with its keyer for superimposition. “Position information2” inFIG. 21is the same.

1-14. Exemplary Delivery Control Conducted by the Controller

A flowchart shown inFIG. 22partially depicts delivery control conducted by the controller110according to the playlist. In Step ST1, the controller110starts delivery control of one formation in the playlist including the camera image, and then goes to the process in Step ST2. In Step ST2, the controller110reads the individual values and the individual settings out of the formation of interest in the playlist. In this case, from the playlist, the controller110acquires camera control parameters, a superimposition image, a target position, a target M/E bank, data (instruction) of a use keyer number, and an input (camera) selected for a background image.

Subsequently, in Step ST3, the controller110sends the camera control parameters to the camera46. Here, the camera control parameters are information to specify the shooting state of the camera46(such as the position, the angle, and the zoom value).

Subsequently, in Step ST4, the controller110receives information about the shooting state (such as the position, the angle, and the zoom value) sent from the camera46. In Step ST5, then, the controller110calculates the shooting area of the camera46from the position, the angle, and the zoom value, and in Step ST6, the controller110determines whether the target is within the shooting area of the camera46.

If the target is within the shooting area, in Step ST7, the controller110calculates the target position in the picture area from the position, the angle, and the zoom value, and calculates the position at which the superimposition image is to be superimposed based on the calculated result. Then, in Step ST8, the controller110instructs the effects switcher to place the superimposition image on the superimposed position with the use keyer.

After the process in Step ST8, the controller110ends delivery control for one formation in Step ST9. In addition, if the target is not within the shooting area of the camera46in Step ST6, the controller110immediately goes to Step ST9, and ends delivery control for one formation.

A flowchart shown inFIG. 23depicts exemplary delivery control conducted by the controller110in response to an operator's manual operation after finishing control for one formation in the playlist. Here, manual operations may be made from the manipulation console41A of the effects switcher41, or may be made from the system control console111of the controller110.

In Step ST11, the controller110starts delivery control including the camera image, and then enters the wait state for manipulation in Step ST12. If a manipulation is made in Step ST12, the controller110determines whether the background image is selected in Step ST13. If the background image is not selected, the controller110returns to Step ST12, and enters the wait state for manipulation.

If the background image is selected, in Step ST14, the controller110determines whether it is a manipulation to select the camera image specified in the playlist. If the specified camera image is selected, in Step ST15, the controller110calculates the shooting area of the camera46from the position, angle, and zoom value of the camera46, and in Step ST16, the controller110determines whether the target is within the shooting area of the camera46.

If the target is within the shooting area, in Step ST17, the controller110calculates the target position in the picture area from the position, the angle, and the zoom value, and calculates the position at which the superimposition image is to be superimposed based on the calculated result. Then, in Step ST18, the controller110instructs the effects switcher41to place the superimposition image on the superimposed position with the use keyer. After the process in Step ST18, the controller110returns to Step ST12, and enters the wait state for manipulation.

If the specified camera image is not selected in Step ST14, and if the target is not within the shooting area in Step ST16, in Step ST19, the controller110instructs the effects switcher41to cancel the placing of the superimposition image. After that, the controller110returns to Step ST12, and enters the wait state for manipulation.

In addition, because “to select the background image” is to select the background image used for outputs in the effects switcher41, this indicates manipulations related to the background bus (Bkgd A) called a bus A. Depending on the types and settings of the effects switcher41, this may be a bus having another name.

In the example described above, such an example is shown that a single camera is specified in one formation in the playlist. However, as another example, a plurality of cameras may be targets. When a plurality of cameras is targets, the controller110is configured to have information about superimposition images to be placed on a plurality of the images taken by the cameras in the playlist, or in a different storage unit.

For example, a superimposition setting table shown inFIG. 24is set in the playlist, or in a different storage unit. The superimposition setting table is stored with the target camera number, the input number of the effects switcher41connected to the video signal of the target camera, and the input number of the effects switcher41connected to the image that is to be superimposed, for each of a plurality of the cameras. In addition, the superimposition setting table is stored with the keyer number for use, the detail to specify video signal processing of the use keyer (such as the control value), and position information (size information is sometimes included) about the target (target subject), for each of a plurality of the cameras.

In addition, when the video signals of the superimposition image is obtained from the material server, the column “the input number of the superimposition image” in the superimposition setting table shown inFIG. 24is “the identifier of the material clip as the superimposition image”, for example. In this case, in operations, which input the effects switcher41selects for a predetermined keyer is determined based on the correspondence between the material clip (identifier) sent from the material server and the output channel.

A flowchart shown inFIG. 25depicts exemplary delivery control conducted by the controller110in response to an operator's manual operation when the superimposition setting table is set.

In Step ST21, the controller110starts delivery control including the camera image, and then enters the wait state for manipulation in Step ST22. If a manipulation is made in Step ST22, the controller110determines whether the background image is selected in Step ST23. If the background image is not selected, the controller110returns to Step ST22, and enters the wait state for manipulation.

If the background image is selected, in Step ST24, the controller110determines whether this is a manipulation to select any one of camera images stored in the superimposition setting table. If a stored camera image is selected, in Step ST25, the controller110reads setting information related to the camera stored in the superimposition setting table, and prepares for use in the processes below.

Subsequently, in Step ST26, the controller110calculates the shooting area from the position, the angle, and the zoom value received from the camera46, and in Step ST27, the controller110determines whether the target is within the shooting area of the camera46.

If the target is within the shooting area, in Step ST28, the controller110calculates the target position in the picture area from the position, angle, and zoom value of the camera46, and calculates the superimposed position at which the specified superimposition image is to be placed based on the calculated result. Then, in Step ST29, the controller110instructs the effects switcher41to place the superimposition image on the superimposed position with the specified keyer. After the process in Step ST29, the controller110returns to Step ST22, and enters the wait state for manipulation.

If a stored camera image is not selected from the superimposition setting table in Step ST24, and if the target is not within the shooting area in Step ST27, the controller110instructs the switcher41to cancel the placing of the superimposition image in Step ST30. After that, the controller110returns to Step ST22, and enters the wait state for manipulation.

In addition, in delivery control conducted by the controller110inFIGS. 23 and 25, preferably, control is conducted such that superimposition is conducted or cancellation of superimposition is conducted at the same timing as the timing of the operation of selecting the background image in the effects switcher41(the field for execution). Thus, because switching the background image and changing superimposition are performed at the same timing, an output image can be obtained with no response delay.

1-15. Delivery Control when the Shooting State of the Camera is Changed

When the shooting state such as the position, the angle (directions of pan and tilt), and the zoom value is changed in the camera46by manual operations, the state of the selected image taken by the camera as the background image is changed. For example, the target position is changed in the taken image. In addition, for example, the target positioned in the taken image is not positioned in the taken image. In addition, for example, the target not included in the taken image is positioned in the taken image. In the embodiment, the controller110is configured to change the superimposition operation conducted by the effects switcher41in accordance with the change in the shooting state of the camera46.

A flowchart shown inFIG. 26depicts exemplary delivery control conducted by the controller110when receiving information about the shooting state from the camera in which the shooting state such as the position, the angle, and the zoom value is changed by manual operations.

In Step ST31, when receiving information about the shooting state from the camera46, the controller110starts delivery control. Then, in Step ST32, the controller110determines whether the source of information about the shooting state is the camera46whose taken image is currently used for the background image. If the source of information about the shooting state is not the camera46whose taken image is currently used for the background image, the controller110immediately goes to Step ST38, and ends the control process.

If the source of information about the shooting state is the camera46whose taken image is currently used for the background image, the controller110goes to the process in Step ST33. In Step ST33, the controller110determines whether superimposition is set in the playlist and in the superimposition setting table as corresponding to the camera46. If superimposition is not set, the controller110immediately goes to Step ST38, and ends the control process.

If superimposition is set, the controller110goes to the process in Step ST34. In Step ST34, the controller110calculates the shooting area of the camera46from information about the received shooting state (such as the position, the angle, and the zoom value), and in Step ST35, the controller110determines whether the target is within the shooting area of the camera46.

If the target is within the shooting area, in Step ST36, the controller110calculates the target position in the picture area from the position, the angle, and the zoom value, and calculates the superimposed position at which the specified superimposition image is to be placed based on the calculated result. Then, in Step ST37, the controller110instructs the effects switcher41to place the superimposition image on the superimposed position with the specified keyer. After the process in Step ST37, the controller110ends the control process in Step ST38.

If the target is not within the shooting area in Step ST35, the controller110instructs the effects switcher41to cancel the placing of the superimposition image in Step ST39, and after that, the controller110goes to Step ST38, and ends the control process.

As discussed above, in the system100shown inFIGS. 1 and 13, when the effects switcher41selects an image taken by the camera46as a background image, the controller110determines whether a superimposition image is to be superimposed based on the shooting state of the camera46. Then, based on the determined result, the controller110instructs the effects switcher41to perform superimposition.

In this case, in order to superimpose the superimposition image on the image taken by the camera46, a portion of the circuit is not exclusively used in the effects switcher41, a portion of the circuit is used as necessary. In addition, in this case, it is unnecessary to provide a superimposition device for cameras separately from the effects switcher41. Therefore, superimposition according to the shooting state of the camera46can be economically implemented without causing degradation of the overall functions.

In addition, in the system100shown inFIGS. 1 and 13, when the effects switcher41selects an image taken by a predetermined camera46among a plurality of the cameras46, it is determined whether a superimposition image is to be superimposed on the image taken by the camera46based on information about the shooting state of the camera46. Then, based on the determined result, the effects switcher41is instructed to superimpose the superimposition image on the image taken by a predetermined camera46. Therefore, even though a plurality of the cameras46is operating, suited superimposition can be economically implemented.

In addition, in the system100shown inFIGS. 1 and 13, the controller110calculates the position at which the superimposition image is to be superimposed based on information about the position, angle, and zoom value of the camera46, and the controller110instructs the effects switcher41of the position at which the superimposition image is to be superimposed based on the calculated result. In this case, because adjusting the superimposed position of the image is automated, the staff necessary to operate a broadcasting system can be downsized to curtail costs, and the possibilities of occurrence of operating error caused by manual operations of superimposed positions can be reduced.

2-1. Other Examples of the Positions at which the Superimposition Image is to be Superimposed

In addition, in the embodiment described above, for the superimposition image placed on the image taken by the camera46, the descriptions are made in which the target position (target subject) in the picture area is calculated, and the superimposition image is arranged at the lower end of the target position, for example. However, the position at which the superimposition image is to be superimposed may be fixed at a certain position in the picture area. As described above, the superimposed position is fixed to a certain position, thereby eliminating the process of finding the target position in the picture area to determine the superimposed position, and making the process easier.

FIGS. 27A to 27Ishow exemplary superimposed positions.FIGS. 27A to 27Care examples that a superimposition image is arranged at the lower end of the target position. In contrast to this,FIGS. 27D to 27Fare examples that a superimposition image is arranged at the center of the picture area, not depending on the target position. In addition,FIGS. 27G to 27Iare examples that a superimposition image is arranged on the lower left in the picture area, not depending on the target position.

The embodiment of the invention can economically implement superimposition according to the shooting state of the camera without causing degradation of the overall functions in relation to a system using an effects switcher, which can be applied to a broadcast programming delivery control system.