Multicast meeting unit and multicast meeting program

The invention provides a television meeting unit that is capable of displaying a picture smoothly on a middle participation terminal. To realize such television meeting unit, the multicast meeting unit is provided with a picture coding means for coding an input picture, a picture transmission means for transmitting the coded picture by mean of multicast, a picture receiving means for receiving the transmitted coded picture, a picture decoding means for decoding the received picture, a picture display means for displaying the decoded picture, a middle participation notice means for notifying middle participation in a meeting, and a middle participation terminal detection means for detecting the middle participation notice and for transmitting it to the picture coding means. The picture coding means, upon receiving the middle participation notice from the middle participation terminal detection means, changes the coding format to the in-frame coding format.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a television meeting unit, a television meeting system, and a method for controlling a television meeting system, and particularly relates to a television meeting unit, a television meeting system, and a method for controlling a television meeting system capable of shortening the time required to display a picture of meeting participants to a new participant who participates in the middle of a meeting.

2. Description of Prior Art

Heretofore, a unit for multiplying the picture and voice at multiple points, namely so-called MCU (Multipoint Control Unit), is needed to hold a television meeting between multiple points.

However, with development of the Internet recently, the group broadcast function so-called multicast has been available, and as the result it becomes possible by use of the multicast to hold an electronic meeting between multiple points without preparation of a particular unit such as MCU.

FIG.1andFIG. 2show a typical exemplary structure of a multicast television meeting system that uses the picture and voice.FIG. 1shows a connection diagram to a network, andFIG. 2is a functional block diagram for showing a schematic structure of terminal unit. The picture and voice generated from the respective terminal units12-1to12-x are transmitted broadcasterly to the respective terminal units12-1to12x, and the received picture and voice are displayed individually or in the synthesized form to realize the television meeting.

The picture and voice of a television meeting system in the form of compressed code are transmitted usually because of the communication capacity restriction. Typically, H. 261 format or MPEG1 format is used as the picture compression coding system. In these picture compression coding systems, the in-frame coded picture in which the pixel value is coded as it is and the inter-frame prediction coded picture are used with switching selectively. The time redundancy of the picture with slight motion and change is reduced by use of the inter-frame prediction coded picture in which the difference between the previous frame and the next frame is coded because the next frame is very similar to the previous frame. On the other hand, in the case of the picture with quick motion or scene change, the in-frame coded picture is used effectively in which the pixel value is coded as it is used effectively because the correlation between frames is slight. In the general television meeting, the in-frame coded picture is transmitted at first and then the inter-frame coded picture is transmitted thereafter because the motion is not so quick.

At first in the case that a participant terminal participates in the middle in the conventional multicast meeting unit, the new participant receives the inter-frame prediction coded picture that is the meeting data served up to that time. Therefore, the terminal that participates in the middle and receives the inter-frame prediction coded picture cannot display a normal picture because no reference picture (in-frame coded picture) is available, or cannot display the picture until it receives an in-frame coded picture.

At second in the case that the processing capability of the new participant terminal is different, it is required for every participant terminal to perform with a set value within the processable range and furthermore it is required to change the set value due to the middle participation.

SUMMARY OF THE INVENTION

To solve the above-mentioned problem, the present invention is provided with a function that a middle participation information is transmitted when a terminal (middle participation terminal) participates in a meeting in the middle, and other terminals (already participating terminal) receive and detect the information and transmit the in-frame coded picture (I-frame) instead of the inter-frame prediction coded picture when the already participating terminals transmit the data. Thereby, it becomes possible to display a picture smoothly (a picture of a meeting can be displayed in a short time) to the terminal that participates in the middle and the multiple point television meeting is held smoothly by use of a multicast meeting unit.

Furthermore, the present invention is provided with a function that the processing capability information on terminals that are participating in a meeting and terminals that are to participate in the meeting is held, collated, and converted to the coded parameter that is processable by any terminal. Thereby, the information is converted so as to be processable by any terminal in the case that it is required to convert the coding parameter due to the middle participation.

Furthermore, the present invention is provided with a function that the priority adding function to terminals that precipitate in a meeting and terminals that are to participate in the meeting, and the coding parameter of the important data is converted. Thereby, the individual terminals are coded correspondingly to the priority, and the important subject is discussed preferentially in the meeting.

Furthermore, the present invention is provided with a function that the priority of the terminals are managed so as to avoid the band over due to I-frame broadcast transmission by adjusting the middle participation and the terminal performance difference to transmit I-frame from the terminal of the higher priority. Thereby, a picture of the terminal of the higher priority is displayed first, and the picture of other meeting participants is displayed later, and as the result a picture of the terminal of higher priority can be displayed earlier without adverse effect on the band of the network.

Therefore, the first object of the present invention is to shorten the time for waiting the display of a picture received by the middle participation terminal in a multicast meeting unit.

The second object of the present invention is to make the coding parameter acceptable to every terminal even in the case that the coding parameter, which has been used by every terminal, becomes unacceptable to a part of terminals due to the middle participation in the meeting.

The above-mentioned objects and advantages of the present invention will be apparent based on the embodiments described in detail hereinafter with reference to the drawings.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be described in detail with reference toFIG. 3to FIG.14. The present invention is by no means limited to these embodiments, and can be applied with various modifications without departing from the spirit and the scope of the present invention.

At first, the configuration for displaying a picture smoothly on a terminal that participates in a meeting in the middle will be described hereinafter.FIG. 3is a functional block diagram showing the schematic structure of the first embodiment of a multicast television terminal unit (simply referred to as “terminal unit” hereinafter depending on the case) of the present invention. InFIG. 3,100denotes a terminal unit, and the terminal unit is connected to a network and used like the system shown in FIG.1. The terminal unit100supplies a picture data obtained from a picture input means101to a picture coding means102, the picture coding means102compresses and codes the supplied picture data and supplies it to a picture transmission means103, and the picture transmission means103transmits the compressed and coded picture data to the network. At that time, the picture coding means102changes the coding method when it obtains a middle participation information from a middle participation detection means113.

Similarly, the terminal unit100supplies the voice data obtained from a voice input means104to a voice coding means105, the voice coding means105compresses and codes the supplied voice data and supplies it to a voice transmission means106, and the voice transmission means106transmits the compressed and coded voice data to the network. A picture receiving means107receives the compressed and coded picture data from the other terminal unit and supplies it to a picture decoding means108. A plural number of picture decoding means108corresponding to the number of other terminal units are provided, each picture decoding means108codes the compressed and coded picture data so as to match with a corresponding terminal and supplies it to a picture display means109. The number of picture display means109corresponds to the number of other terminal units. Each picture display means109displays a picture of a corresponding terminal.

Similarly, a voice receiving means110receives the compressed and coded voice data from the other terminal unit, and supplies it to a voice decoding means111. The number of voice decoding means111corresponds to the number of other terminal units. Each voice decoding means111decodes the compressed and coded voice data for a corresponding terminal, and supplies it to a voice output means112. The number of voice output means112corresponds to the number of other terminal units. Each voice output means112outputs the voice of a corresponding terminal. A middle participation notice means114supplies a middle participation notice to the network when a terminal unit participates in the middle.FIG. 4shows an example of a picture data packet to which a transmitter information has been added that is transmitted by the middle participation notice means114. An example of the middle participation notice data, which is transmitted when a middle participation is notified, includes the data in which an address information on a transmittee and a transmitter information on a middle participation terminal are placed adjacently.

FIG. 5is a functional block diagram showing the schematic structure to which a reception notice detection means is added. InFIG. 5,116denotes a reception notice transmission means for transmitting a reception notice when a middle participation terminal receives an in-frame coded picture from the other meeting terminal, and115denotes a reception notice receiving means of a (the other) meeting terminal for receiving a reception notice that has been transmitted from a middle participation terminal through the reception notice transmission means116(described in detail hereinafter).

FIG. 6is a functional block diagram showing the schematic structure to which a network error rate measurement means has been added. InFIG. 6,117denotes a network error rate measurement means for measuring the network error rate when the picture data is obtained from the picture receiving means107(described in detail hereinafter).

FIG. 7shows a method of middle participation notice that uses the multicast, and FIG.8A andFIG. 8Bshow methods of middle participation notice that use the unicast. Furthermore, FIG.9A andFIG. 9Bshow coding formats that code a picture signal transmitted from the picture coding means102. Out of these figures,FIG. 9Ais a diagram for showing a coding format of a transmitted picture signal that is used conventionally and is used when a multicast meeting is continued without a middle participation in the present invention. Furthermore,FIG. 9Bis a diagram for showing a method for changing the P-picture coding format to the I-picture coding format when a middle participation notice is received during the middle of a meeting, which is employed in the present invention. In FIG.9A andFIG. 9B, I denotes the I-picture (in-frame coded picture), and P denotes the P-picture (inter-frame forward prediction coded picture).

The operation will be described herein under on the terminal unit having the above-mentioned structure. The picture data supplied from the picture input means101such as a camera is supplied to the picture coding means102. The picture coding means102changes the coding format as described hereinafter based on the information supplied from the middle participation means113, and compresses and codes the picture data. The compressed and coded picture data is transmitted to the network from the picture transmission means103by use of the multicast.

The voice data supplied from the voice output means104such as a microphone is supplied to the voice coding means105. The voice coding means105compresses and codes the voice data. The compressed and coded voice data is transmitted to the network from the voice transmission means106by use of the multicast. The picture data transmitted from the other terminal unit is received by the picture receiving means107through the network. The picture receiving means107supplies the received picture data to the picture decoding means108-1˜n corresponding to the respective other terminal units with reference to the transmitter information on the picture data. One example of the transmitter information that is embedded in the data to be transmitted is shown in FIG.4. The number of picture decoding means108-1˜n corresponds to the number of other terminal units. Each picture decoding means decodes the picture data to be transmitted to the picture display means109-1˜n. The number of picture display means109-1˜n corresponds to the number of other terminal units. Each picture display means displays the decoded picture data on a display.

The voice data transmitted from the other terminal unit is received by the voice receiving means110through the network. The voice receiving means110supplies the received voice data to the voice decoding means111-1˜n corresponding to the respective other terminal units with reference to the transmitter information on the voice data in the same manner as used in the case of the picture data. The number of voice decoding means111-1˜n corresponds to the number of other terminal units. Each voice decoding means decodes the voice data and supplies it to the voice output means112-1˜n. The number of voice output means112-1˜n corresponds to the number of other terminal units. Each voice output means outputs the decoded voice data from a speaker. Otherwise, a method, in which one terminal is provided with not a plurality of voice output means, but one terminal is provided with one voice output means by providing a voice synthesis means so-called mixer between the voice decoding means and the voice output means, may be employed.

When the terminal unit participates in a meeting in the middle, the middle participation notice means114notifies the other terminals of the middle participation through the multicast or unicast.

In the case that the middle participation is notified through the multicast, the middle participation notice means114notifies the other terminals of the middle participation simultaneously as shown in FIG.7. On the other hand, in the case that the middle participation is notified through the unicast, two methods may be available as shown in FIG.8. In the one method, the address of meeting terminals that are participating in a current meeting has been obtained previously and the middle participation is supplied individually as shown in FIG.8A. In the other method, the middle participation is supplied to any one meeting terminal among meeting terminals that are currently participating in a meeting and the meeting terminal that has received the middle participation propagates it to the other meeting terminal successively as shown in FIG.8B.

Methods shown in FIG.9A andFIG. 9Bare exemplified as the method for changing the coding format in the picture coding means102.

In the case of the conventional transferred picture coding format as shown inFIG. 9A, a middle participation terminal receives a P-picture because terminals that are participating in a meeting currently transmit the P-picture when the middle participation terminal participates in the meeting in the middle. The middle participation terminal that receives the P-picture cannot decode a normal picture because the middle participation terminal has no I-picture, which is the reference picture.

However, it is possible to change the coding method when a middle participation terminal participates in a meeting in the middle because current participation terminals receive a middle participation notice. Therefore, the current terminals that are participating in the meeting receives the middle participation notice at the middle participation detection means113when the middle participation terminal participates in the meeting in the middle, and the information is supplied to the picture coding means102. In response to the information, the picture coding means102that has continued coding to P-picture until now changes the coding format to I-picture that can be decoded singularly.

Three I-picture sending terms are available as described herein under.

In the first method, the I-picture is sent for a certain time. One frame picture data of changed coding method or a certain number of frames of changed coding method is sent out, and the coding method is changed to the original coding method thereafter.

In the second method, when the middle participation terminal receives an in-frame coded picture from other meeting terminals, the middle participation terminal issues a reception notice to other meeting terminals, and other meeting terminals change the coding method to the original coding method. The second method will be described with reference to FIG.5.

When the picture receiving means107of the middle participation terminal receives an I-picture from other meeting terminals, the information for notifying the reception is supplied to the reception notice transmission means116. The reception notice transmission means116transmits a reception notice to other meeting terminals by use of the multicast as shown inFIG. 7or by use of the unicast as shown in FIG.8A. When the unicast is used, the transmitter information shown inFIG. 4included in the received I-picture is used to specify the transmitter.

Other meeting terminals receive the reception notice through the reception notice receiving means115, the information for notifying reception of the reception notice is supplied to the picture coding means102, and the picture coding means102changes the picture coding format to the original format.

Because other meeting terminals can recognize that the middle participation terminal has received the reference picture (I-picture), other meeting terminals can continue to transmit the reference picture until the middle participation terminal receives the reference picture.

In the third method, the network error rate is measured, other meeting terminals continue to use the changed coding format and transmit the reference picture for only the time corresponding to the error rate. The third method will be described with reference to FIG.6.

The network error rate measurement means117measures the error rate in the picture data received through the picture receiving means107based on, for example, the sequence number embedded in the picture data, and the measured error rate is supplied to the picture coding means102. The picture coding means102determines the reference frame sending term shown inFIG. 9depending on the error rate supplied from the network error rate measurement means117. In detail, the doubled reference frame sending term may be employed for the doubled error rate. For example, if the reference picture of two frames is transmitted at the time of the middle participation under the condition that one packet error is included in 100 packets, the reference picture of doubled, namely four, frames is transmitted at the time of the middle participation when the error rate is doubled (the condition that two packet errors are included in 100 packets).

Because the picture receiving means107receives the picture data from a plurality of meeting terminals, the mounting method that satisfy the highest error rate may be employed. It is apparent that this method may be used for the middle participation terminal and other meeting terminals. Thereby, a plurality of I-picture, namely reference picture, can be sent out in the case that the network error rate is high, the probability of reception of the reference picture by the middle participation terminal is improved with minimized load on the network. By using the structure described hereinbefore, it is possible to display the picture of a meeting within a short time by use of the multicast television meeting unit even though a terminal participates in the meeting in the middle, and the multiple point television meeting can be continued smoothly.

Next, a mechanism for exchanging the processing capability information on each terminal so that all the terminals participating in a meeting can display the picture and a mechanism for giving the priority to the data that is important in the meeting will be described.FIG. 10is a functional block diagram showing the schematic structure of the second embodiment of a multicast television meeting terminal unit of the present invention. InFIG. 10,800denotes a terminal unit, and the terminal unit is connected to a network and used.

The terminal unit800supplies the picture data obtained from a picture input means801to a picture coding means802, the picture coding means802compresses and codes the supplied picture data and supplies it to a picture transmission means803, and the picture transmission means803transmits the compressed and coded picture data to the network. At that time, the picture coding means802obtains a middle participation information from a picture receiving means807and changes the coding method. The terminal name of the middle participation terminal and the picture data (coding format, parameter) transmitted from the middle participation terminal are held in a coding state holding means813.

Similarly, the voice data obtained from a voice input means804is supplied to a voice coding means805, the voice coding means805compresses and codes the supplied voice data and supplies to a voice transmission means806, and the voice transmission means806transmits the compressed and coded voice data to the network. At that time, the information such as voice level is supplied from the voice input means804to an significance information adding means814. Furthermore, the voice coding means805obtains the voice coding format or voice coding parameter from a voice receiving means810and changes the coding method. Furthermore, the picture receiving means807receives the compressed and coded picture data from the other terminal unit through the network, and supplies it to a picture decoding means808. At that time, the received picture information is supplied to the picture coding means802. The number of picture decoding means808corresponds to the number of other terminal units, and the compressed and coded picture data is decoded so as to match with each terminal and supplied to picture display means809. The number of picture display means809corresponds to the number of other terminal units. The picture display means displays a picture of each terminal.

Similarly, the voice receiving means810receives the compressed and coded voice data from the other terminal unit through the network, and supplies it to a voice decoding means811. The number of voice decoding means811corresponds to the number of other terminal units, and the compressed and coded voice data is decoded so as to match with each terminal and supplied to a voice output means812. The number of voice output means812corresponds to the number of other terminal units. The voice output means outputs the voice of on each terminal.

FIG. 11shows one example of a picture data packet that is to be transmitted to which the significance information is added.FIG. 12shows an example of the structure of a table for holding the picture coding information, the state of middle participation and middle withdrawing, and the picture coding format used in the meeting at that time.

The operation of the terminal unit having the structure described hereinabove will be described herein under.

The picture data taken in from the picture input means801such as camera is supplied to the picture coding means802, and the picture data is compressed and coded.

A method described hereinunder, in which the middle participation is detected by the picture receiving means807without using the middle participation notice means114as in the case of the embodiment 1 when the picture coding means802compresses and codes the picture data, may be employed.

The picture receiving means807refers the transmitter information on the received picture data as shown in FIG.4. If the transmitter information is a transmitter information on a terminal other than current participation terminals, the transmitter information is judged as that of a middle participation terminal, and the information for notifying participation of the middle participation terminal is supplied to the picture coding means802.

As an exemplary “method for discrimination between the current participation terminal and the middle participation terminal”, a method in which the transmitter information on the current participation terminals is stored in a memory of each terminal may be employed. In the case that the data having the transmitter information that is not stored in the memory is supplied, it is possible to judge the data to indicate participation of a middle participation terminal.

The middle participation is detected as described hereinabove, and the picture coding means802changes the coding format to the in-frame coded picture (reference picture) as shown in FIG.9. As the result, it is possible to display the picture of other meeting terminals on the middle participation terminal within a short time.

The compressed and coded picture data is transmitted to the network through the picture transmission means803. The picture transmission means803can add the significance to the picture data as shown inFIG. 11when the picture data is transmitted. The significance represents the dynamic significance in the meeting such as a speaker. A method may be employed, in which when the level of significance of a speaker is up, the voice input level supplied from the voice input means804is supplied to the significance information adding means814, and the significance information adding means814adds the significance corresponding to the voice level. The picture quality is improved, the frame rate is increased, or the picture size may be changed by use of the picture coding means802for the case of the picture data of high significance.

For example, the processing is performed normally at a performance value (parameter): B that is lower than the performance value (parameter): A of the terminal having the lowest performance based on the processing capability information exchange (<that will be described hereinafter as<parameter setting relevant to the processing capability of the terminal>) with emphasis on the saving of NW band and immediate data processing, and only when the data to which the significance is added comes, the level of the performance value is up to a higher value within a range lower than the performance value A for the data with significance. As described hereinabove, the quality of the picture supplied from a terminal that is considered to be important such as a speaker in the meeting is changed dynamically, and as the result the quality of a picture that attracts attentions of participants is improved. The voice data supplied from the voice input means804such as a microphone is supplied to the voice coding means805. The voice coding means805compresses and codes the voice data. The compressed and coded voice data is transmitted to the network by use of the multicast through the voice transmission means806.

The picture data transmitted from other terminal units is received by the picture receiving means807through the network. The picture receiving means807supplies the received picture data to the picture decoding means808-1˜n corresponding to the respective other terminal unit with reference to the transmitter information on the picture data. The number of picture decoding means808-1˜n corresponds to the number of other terminal units. Each of the picture decoding means808-1˜n decodes the picture data and supplies it to the picture display means809-1˜n. The number of picture display means corresponds to the number of other terminal units similarly, and the decoded picture data is displayed on a display.

The voice data transmitted from the other terminal units is received by the voice receiving means810through the network. The voice receiving means810supplies the received voice data to the voice decoding means811-1˜n corresponding to the respective terminal units with reference to the transmitter information on the voice data. The number of voice decoding means811-1˜n corresponds to the number of other terminal units. Each of the voice decoding means811-1˜n decodes the voice data and supplies it to the voice output means812-1˜n. The number of voice output means812-1˜n corresponds to the number of other terminal units similarly, and the decoded voice data is outputted from a speaker. In some cases a method may be employed, in which one terminal is provided with not a plurality of voice output means, but one terminal is provided with only one voice output means with aid of a voice synthesis means so-called mixer interposed between the voice decoding means and the voice output means.

Examples of the significance addition includes a case in which significance is added depending on the user, that is, the significance is up for the user of the chairman, and a case in which a high significance is added to a terminal displaying the significant picture motion, other than the case in which the high significance is added depending on the input level of the voice. Furthermore, a method in which the picture coding means802changes the picture coding format will be described herein under. A method shown inFIG. 9is exemplified as a method in which the picture coding means802changes the coding format similarly to the embodiment 1.

<Parameter Setting Relevant to Processing Capability of Participation Terminal>

The processing capability information exchange will be described as the premise for adding the above-mentioned significance. A middle participation terminal codes the data by use of the coding format and parameter that are relevant to the processing capability (CPU performance) and supplies it to other meeting terminals. The other meeting terminals that have received it obtain the picture size, the number of frames, and the data quantity of the received picture information, and then changes the coding format so as to fit the received data, and as the result the other meeting terminals can supply the data that is relevant to the processing capability of the middle participation terminal.

A method for changing the coding will be described in detail with reference toFIG. 12A,FIG. 12B, and FIG.12C. At first, in the initial state of a meeting shown inFIG. 12A, terminals A, B, and C are participant in the meeting. In this state, because the picture coding parameter sent from the terminal A is lowest, the picture coding format CIF, 10 fps, and 64 bps, that is the picture coding format of the terminal A, is used as the picture coding format in the meeting. Herein, the picture size CIF represents 352 (pixels)×288 (pixels), and QCIF represents 176 (pixels)×144 (pixels).

Next, the state in which the terminal D (QCIF, 5 fps, 32 kbps) participates in the meeting in the middle is shown in FIG.12B. Herein, because the processing capability of the terminal D is lowest, the picture coding format of the terminal D that is lowest in the processing capability is used in the meeting in the same manner as used previously. Next, the state in which the terminal A and the terminal D withdraw from the meeting in the middle is shown in FIG.12C. The data of the terminal A and the terminal D that have withdrawn from the meeting is deleted from the table, the picture coding information on the terminal C having the lowest processing capability among the residual terminals is used similarly, and the meeting is continued by the residual meeting terminals. The level of the processing capability is defined in terminal units herein. The reason is that all the values of the respective coding information are lower than those of other terminals. In the case that the terminal T1has the picture size of the lowest value t1, the terminal T2has the frame rate of the lowest value t2, and the terminal T3has the bit rate of the lowest value t3, (t1, t2, and t3) are used as the parameter for decision.

H. 323 capacity exchange has been known as a conventional method for determining parameters used by each terminal as described hereinabove. In this method, after each terminal extends the session, parameters that can be encoded and decoded are exchanged each other and a meeting is continued by use of parameters that are usable each other. This method is possible only for 1:1 meeting, and IMCU should be used for using this method in the case that three or more terminals participate. In this case, at first, each terminal transmits parameters that can be encoded and decoded to MCU. Next, MCU detects parameters that can be acceptable for all the terminals based on the parameters supplied from each terminal, and transmits the parameter to each terminal. Each terminal participates in the meeting by use of the parameters that have been detected by MCU.

However, because the structure of the present embodiment does not use MCU, this method is not applied. In the present embodiment, the multicast address is acquired from DHCP server, and the picture and voice are transmitted immediately. The parameter is changed to the parameter that is usable each other on the terminal side that has received the picture and voice. Because each terminal decides the parameter based on the supplied picture and voice as described hereinabove, the structure having no MCU can be employed. Furthermore, in the conventional method, all the terminals should transmit parameters to MCU, MCU re-detects the parameter that is usable for all the terminals and distributes the parameter to each terminal in the case a terminal participates in the meeting in the middle. However, in the case of the present embodiment, these procedures can be eliminated, a middle participation terminal can transmit the picture and voice immediately, and the time required for the middle participation terminal to participate in the meeting is shortened.

The same is applied to the voice data, the middle participation terminal codes the voice data by use of the coding format and parameter that are relevant to the processing capability and supplies it to other meeting terminals, the other meeting terminals that receive it obtain the coding format and coding parameter of the received voice information and change the coding to the relevant coding to thereby supply the data that is relevant to the processing capability of the middle participation terminal. As described hereinabove, in the present embodiment, by changing the coding method to a method that is relevant to the received coding format (change the coding parameter), it is possible to continue a meeting (the meeting data is displayed even on a terminal having the low processing capability) when a meeting terminal having a low processing capability participates in the meeting in the middle. Furthermore, by using the significant information, the high significant picture such as the picture of a speaker is displayed with higher picture quality.

The middle participation detection means113described in the embodiment 1 and the coding state holding means813described in the second embodiment 2 may be used in combination to receive the middle participation notice through the middle participation detection means113. In this case, an exemplary method, in which the current coding information is held in the coding state holding means813and the coding format is changed, may be realized.

Next, a mechanism for avoiding the inconvenience that the band is occupied fully when all the terminals that are participating in a meeting transmit I-frame simultaneously in response to reception of a notice of middle participation will be described herein under.

FIG. 13is a functional block diagram showing the schematic structure of the third embodiment of a multicast television meeting terminal unit of the present invention.

InFIG. 13,1100denotes a terminal unit, and the terminal unit is connected to the network and used as in the case of the system shown in FIG.1. The terminal unit1100supplies the picture data obtained from a picture input means1101to a picture coding means1102, the picture coding means1102compresses and codes the supplied picture data and supplies it to a picture transmission means1103, and the picture transmission means1103transmits the compressed and coded picture data to the network. At that time, the picture coding means1102changes the coding method depending on the priority supplied from the priority decision means1114. The coding information that is used before the change is held in the coding state holding means1113in the same table form as used in the embodiment 2.

The voice data obtained from a voice input means1104is supplied to a voice coding means1105, the voice coding means1105compresses and coded the supplied voice data and supplies it to a voice transmission means1106, and the voice transmission means116transmits the compressed and coded voice data to the network. Furthermore, a picture receiving means1107receives the compressed and coded picture data from the other terminal unit through the network, and transmits it to a picture decoding means1108. At that time, the received picture information is supplied also to the priority decision means1114. The number of picture decoding means1108corresponds to the number of other terminal units, and the compressed and coded picture data is decoded so as to match with each terminal and supplied to the display means1109. The number of picture display means1109corresponds to the number of other terminal units similarly, and the picture of each terminal is displayed.

Similarly, a voice receiving means1110receives the compressed and coded voice data from the other terminal unit through the network, and supplies it to the voice decoding means1111. The number of voice decoding means1111corresponds to the number of other terminal units, and the compressed and coded voice data is decoded so as to match with each terminal and is supplied to the voice out means1112. The number of voice output means1112corresponds to the number of other terminal units, and the voice of each terminal is outputted.

The operation of the terminal unit having the structure as described hereinabove will be described hereinafter. The picture data supplied from the picture input means such as a camera is supplied to the picture coding means1102. The picture coding means1102changes the coding format based on the priority information supplied from the priority decision means1114, and compresses and codes the picture data.

The priority information is prepared by the priority decision means1114that has obtained the picture data information from the picture receiving means1107. For example, a promoter of the meeting assigns the priority to each terminal previously at the time before the meeting is held. In many cases, the terminal of the chairman is assigned a high priority, and a terminal used by a general participant is assigned a low priority. The picture coding means1102changes the picture coding format according to the priority when a middle participation terminal is detected. For example, when a middle participation terminal is detected, other terminals change the coding format from the in-frame prediction coded picture to the inter-frame coded picture as shown in FIG.9. At that time, if all the terminal changes the coding format simultaneously, the picture data quantity transmitted through the network increases sharply to result in pressed band. To avoid the above-mentioned problem, the picture is changed to the in-frame coding picture in the order successively from the terminal having the highest priority for transmission with reference to the priority information. Thereby, the middle participation terminal can display the picture in the order according to the priority.

The compressed and coded picture data is transmitted from the picture transmission means1103to the network by use of the multicast. The voice data supplied from the voice output means1104such as a microphone is supplied to the voice coding means1105. The voice coding means1105compresses and codes the voice data. The compressed and coded voice data is transmitted from the voice transmission means1106to the network by use of the multicast.

The picture data transmitted from the other terminal unit is received by the picture receiving means1107through the network. The picture receiving means1107supplies the received picture data to the picture decoding means1108-1˜n corresponding to the respective other terminal units with reference to the transmitter information on the picture data. The number of picture decoding means1108-1˜n corresponds to the number of other terminal units. Each of the picture decoding means1108-1˜n decodes the picture data and transmits the picture to the picture display means1109-1˜n. The number of picture display means1109-1˜n corresponds to the number of other terminal units similarly, and the decoded picture data is displayed on a display.

The voice data transmitted from the other terminal unit is received by the voice receiving means1110through the network. The voice receiving means1110supplies the received voice data to the voice decoding means1111-1˜n corresponding to the respective other terminal units with reference to the transmitter information on the voice data. The number of voice decoding means1111-1˜n corresponds to the number of other terminal units, and the voice decoding means1111-1˜n decode the voice data and supply it to the voice output means1112-1˜n. The number of voice output means1112-1˜n corresponds to the number of other terminal units similarly, and the decoded voice data is outputted from a speaker. In some cases a method may be employed, in which one terminal is provided with not a plurality of voice output means, but one terminal is provided with only one voice output means with aid of a voice synthesis means so-called mixer interposed between the voice decoding means and the voice output means.

For example, a method shown inFIG. 9that is similar to the method described in example 1 may be exemplified as a method that is used by the picture coding means1102to change the coding format. In the present example, the (a part of) priority information is transmitted from the picture receiving means1107to the priority decision means1114, but the transmission is not necessarily required. For example, (1) the data transmission from1107to1114can be eliminated by holding the priority information previously in the priority decision means114(a dashed line inFIG. 13indicates the case). Furthermore, for example, (2) only the information that indicates whether the I-frame is received or not (whether a middle participation terminal participates or does not) is supplied from the picture receiving means1107to the picture coding means1102, and the coding method is changed immediately or later depending on the information supplied from the priority decision means1114. As described hereinabove, various modifications may be employed.

As described hereinabove, in the present embodiment, by use of the structure described hereinabove, not only it is possible to display the picture data of a meeting within a short time as described in the example 1 but also it is possible that the picture data of the terminal having the higher priority can be displayed earlier without pressing of the band of the network by use of the priority information. In the latter case, for example, the picture data with high priority of, for example, the chairman is displayed earlier and the picture data of other meeting participants are displayed later generally.

In the present embodiment, the unit other than personal computers such as desktop or potable computers can be used as the terminal unit that can participates in a meeting (including middle participation).FIG. 14is a functional block diagram showing the schematic structure of the fourth embodiment of a terminal unit of the present invention. The cellular phone is used as the terminal unit in the present embodiment. InFIG. 14,400denotes a cellular phone served as the terminal unit, and the cellular phone400is connected to the network and used as in the case of the terminal unit100shown in FIG.1. The cellular phone400is provided with a picture input means101for receiving a picture signal, a picture coding means102for receiving the picture signal from the picture input means101and for compressing and coding the picture signal, a picture transmission means103for generating the compressed and coded transmission picture data, and a communication means250for transmitting the compressed and coded picture data generated from the picture transmission means103to the network as in the case of the terminal unit100described in the first embodiment.

Furthermore, the cellular phone400is provided with a voice input means104for receiving a voice signal, a voice coding means105for receiving the voice signal from the voice input means104and for compressing and coding the voice signal, and a voice transmission means106for transmitting the compressed and coded voice data to the communication means250.

Furthermore, the cellular phone400is provided with a picture receiving means107for receiving the compressed and coded picture data supplied from the other cellular phone or terminal unit through the network, picture decoding means108for decoding the received picture data, and picture display means109for displaying the decoded picture. The number of picture decoding means108corresponds to the number of other cellular phones, and the supplied picture data is decoded so as to match with each cellular phone or terminal unit and is supplied to the picture display means109. The number of picture display means109corresponds to the number of other cellular phones similarly, and the picture display means display the picture on the respective terminals.

Furthermore, the cellular phone400is provided with a voice receiving means110, voice decoding means111, and voice output means112for receiving and reproducing the voice transmitted from the other cellular phone or terminal unit. The voice receiving means110receives the compressed and coded voice data from the other cellular phone or terminal unit through the network and supplies it to the voice decoding means111. The number of voice decoding means111corresponds to the number of other cellular phones, and the compressed and coded voice data is decoded so as to match with each terminal and supplied to the voice output means112. The number of voice output means112corresponds to the number of other cellular phones and terminal units similarly. The voice output means outputs the voice of each terminals.

Furthermore, the cellular phone400is provided with a middle participation notice means114that is operated when this cellular phone participates in a meeting in the middle, and a middle participation detection means113for detecting a middle participation notice signal transmitted when the other cellular phone participates in the meeting in the middle as in the case of the terminal unit in accordance with the first embodiment.

The communication250has not only a function to control the communication between the cellular phone400and a base station but also a function to transmit the compressed and coded picture data and compressed and coded voice data to the network and a function to receive the compressed and coded picture data and compressed and coded voice data from the network, namely the two-way function. Furthermore, the communication means250functions to transmit a middle participation notice signal to the network and to receive a middle participation notice signal from the network.

Because the cellular phone400is provided with the above-mentioned function, it is possible for the cellular phone400in accordance with the present embodiment to participate in a multicast meeting. The middle participation notice means114supplies the middle participation information to the communication means250through the network when the cellular phone400participates in the meeting in the middle. On the other hand, the middle participation detection means113receives and detects a middle participation notice signal transmitted from the other cellular phone from the network through the communication means250.

The notice and detection of middle participation in a multicast meeting are operated in the same manner as that operated in the first embodiment. The present invention is applied for potable terminal units as the mobile apparatus such as the cellular phone similarly.

In each embodiment, the in-frame coded “picture” detection units are used as the information for detecting middle participation, but otherwise a method in which the rate of the in-frame coded “macro-block” in the picture is detected as the unit to detect middle participation may be employed. In detail, in the case that the coding format such as MPEG-4 that uses the in-frame coded macro-block is used, not the in-frame coded picture is detected, but the rate of the in-frame coded macro-block in the picture is detected to detect middle participation. For example, the following detection method is exemplified as a method for detecting the in-frame coded macro-block.

At first, the macro-block coding format in the picture is counted. Because the coding format information is stored in each macro-block, the macro-block coding format can be counted easily by referring the coding format information. Next, the rate of the in-frame coded macro-block in the picture is counted, and if the rate is equal to a half or more, then it is judged to be the middle participation.

Furthermore, in each embodiment, the case in which the number of the picture decoding means, picture display means, voice decoding means, and voice output means corresponding respectively to the number of other terminals (hardware or software having such function) are provided is described, but otherwise one hardware or software having such function performs these processing by means of time sharing technique in some cases. Therefore, the number of these means is by no means limited.

Furthermore, in each embodiment, (1) coding method represents the type of codec (CODEC: COmpression/DECompression) such as H. 261, H263, or MPEG, (2) coding format represents the difference between the in-frame coding (I-frame) and the inter-frame coding (P-frame), and (3) coding information (coding parameter) represents the picture size, frame rate, and bit rate. Furthermore, in (3), the coding information is synonymical for the coding parameter, but in some cases the coding information is used to represent the item of attribute and the coding parameter is used to represent the item of attribute value differently (difference between the attribute and attribute value).

The present invention is described based on the preferred embodiments shown in the drawings, but it will be obvious to those skilled in the art that the present invention can be easily modified and changed in various ways without departing from the spirit and the scope of the invention. The present invention includes such various modifications.