Communication device and data processing method

A packet reception state monitoring unit calculates a target stored data number as a target number of audio packets stored in a memory based on the variation in the reception intervals of audio packets, and a reproduction control unit controls the audio reproduction speed in accordance with the difference between the target stored data number and a memory-stored data number. When the memory-stored data number is smaller than the target stored data number, reproduction control is performed in a low-speed reproduction mode at a lower speed than a regular reproduction speed. When the memory-stored data number is larger than the target stored data number, reproduction control is performed in a high-speed reproduction mode at a higher speed than the regular reproduction speed. A reproduction speed changing process is performed, with hysteresis being given to transitions of the difference between the memory-stored data number and the target stored data number.

TECHNICAL FIELD

The present disclosure relates to a communication device and a data processing method. More particularly, the present disclosure relates to a communication device and a data processing method for stably outputting data that is received via a network.

BACKGROUND ART

The use of so-called cloud computing systems is becoming popular these days. In a cloud computing system, a data process request is transmitted from a client such as a portable terminal to a server, data processing is performed at the server, the result of the data processing is returned to the client, and the client obtains the result of the data processing.

An example of the use of such a cloud system is a game played on a portable gaming machine. The portable gaming machine (a client) being operated by a user communicates with a server, and image data such as a game development image and audio data are provided to the client with the use of a highly-sophisticated data processing function of the server.

In such a cloud-type gaming system, user action information is transmitted from the client to the server. In accordance with the action information received from the client, the server develops the game, generates updated images and audio data, and transmits those images and data to the client. The client reproduces the updated images and sound received from the server. By conducting such a process, even a client that does not have a sophisticated processor or the like can obtain results of data processing performed with the use of a highly-sophisticated data processing function of the server.

However, in the data being transferred between the server and the client, delays or jitter (fluctuations) occurs due to a network condition. Normally, image and audio data is transmitted as encoded data, and it is necessary for the client to sequentially decode and output the received data.

When delays or jitter (fluctuations) occurs due to a network condition, the series of processes such as packet reception, decoding, and display on the client side are disturbed, and a discontinuous image or intermittent sound might be generated.

As a technique to solve problems such as intermittent sound due to delays of communication data or the like, a process of changing the audio reproduction speed in accordance with packet delays is disclosed in U.S. Pat. No. 8,560,331 B1.

SUMMARY

Technical Problem

In a case where a process of outputting data such as images and sound received via a network is performed as in the above described cloud-type system or the like, for example, stable data outputting might be hindered by a communication delay.

As for a structure to avoid unstable data outputting, the above mentioned Patent Literature, U.S. Pat. No. 8,560,331 B1, discloses a structure that monitors a delay of each packet and controls the reproduction speed in accordance with the delay state of each packet.

If the reproduction speed is changed in accordance with the delay state of each packet, however, the reproduction speed often varies, and might give viewers a feeling of strangeness.

Also, to change the reproduction speed in accordance with the delay state of each packet, reproduction speed control is performed frequently, and as a result, the load applied to the device becomes too large.

The present disclosure is made in view of those circumstances, and is to provide a communication device and a data processing method for realizing stable data outputting by performing smooth reproduction speed control, with no viewers having a feeling of strangeness and no excessive load being applied to the device.

Solution to Problem

A first aspect of the present disclosure is a communication device that includes: a packet reception state monitoring unit that monitors reception intervals of packets received by a communication unit, and generates and outputs a target value as a parameter for a reproduction control process based on the variation in the packet reception intervals; and a reproduction control unit that performs reproduction control in accordance with the target value generated by the packet reception state monitoring unit, wherein the reproduction control unit controls an audio reproduction speed or image frame reproduction in accordance with the target value.

A second aspect of the present disclosure is a data processing method that is implemented in a communication device, includes: a packet reception state monitoring unit monitoring reception intervals of packets received by a communication unit, and generating and outputting a target value as a parameter for a reproduction control process based on the variation in the packet reception intervals; and a reproduction control unit performing reproduction control in accordance with the target value generated by the packet reception state monitoring unit, wherein an audio reproduction speed or image frame reproduction is controlled in accordance with the target value in the reproduction control.

Other objects, features, and advantages of the present disclosure will become apparent in conjunction with the detailed description of embodiments of the present disclosure and the accompanying drawings. In this specification, a “system” means a logical assembly of devices, and the respective devices are not necessarily located in the same housing.

Advantageous Effects of Invention

With the structure of an embodiment of the present disclosure, a device and a method for monitoring the variation in packet reception intervals and performing optimum reproduction control in accordance with the variation are realized.

Specifically, a packet reception state monitoring unit calculates a target stored data number as a target number of audio packets stored in a memory based on the variation in the reception intervals of audio packets, and a reproduction control unit controls the audio reproduction speed in accordance with the difference between the target stored data number and a memory-stored data number. When the memory-stored data number is smaller than the target stored data number, reproduction control is performed in a low-speed reproduction mode at a lower speed than a regular reproduction speed. When the memory-stored data number is larger than the target stored data number, reproduction control is performed in a high-speed reproduction mode at a higher speed than the regular reproduction speed. A reproduction speed changing process is performed, with hysteresis being given to transitions of the difference between the memory-stored data number and the target stored data number.

With this structure, a device and a method for monitoring the variation in packet reception intervals and performing optimum reproduction control in accordance with the variation is realized.

It should be noted that the effects described in this specification are merely example and do not limit the disclosure, and additional effects may be achieved.

DESCRIPTION OF EMBODIMENTS

The following is a detailed description of a communication device and a data processing method of the present disclosure, with reference to the accompanying drawings. Explanation will be made in the following order.

1. Example structure of a communication system

2. Example structure of a client device

3. Details of a reproduction speed control process to be performed by the audio data processing unit

4. Specific example of the audio reproduction speed control process

5. Sequence of the reproduction speed control process to be performed by the audio data processing unit

6. Example of an image data reproduction control process to be performed by the image data processing unit

7. Sequence of the reproduction control process to be performed by the image data processing unit

8. Example hardware configuration of the client

9. Summary of the structures of the present disclosure

1. Example Structure of a Communication System

Referring first toFIG. 1, an example structure of a communication system having a client that performs a process of the present disclosure is described.

As shown inFIG. 1, a communication system10includes a client20and a server30that can perform bidirectional communications. The client20and the server30perform communications via a network such as the Internet.

The client20is a user terminal device for a general user, and more specifically, is a television receiver21, a PC22, or a portable terminal23such as a game machine or a smartphone, for example.

The client20transmits user action information about the client20to the server30. The server30performs data processing in accordance with the action information received from the client20. In a case where the user is playing a game at the client20, for example, the game is developed in accordance with user actions, and stream data generated by encoding updated image and audio data is transmitted to the client20.

The client20decodes the stream data received from the server30, and outputs an image and sound as a decode result via a display and a speaker.

2. Example Structure of the Client Device

Referring now toFIG. 2, an example structure of the client20as a communication device that performs a communication process with the server30is described.

As shown inFIG. 2, the client20includes a data processing unit51, a communication unit52, a storage unit53, an input unit54, and an output unit55.

The communication unit52receives data distributed from the server30, such as a packet storing images, sound, and the like.

The data processing unit51acquires image data and audio data from the received packet, and performs a decoding process and the like, to generate and output reproduction data.

User instructions and the like are input via the input unit54.

The reproduction data is output to the output unit55including a display unit, a speaker, and the like.

The storage unit53stores the received data including images, sound, and a subtitle.

The storage unit53is also used as the work area for the data processing to be performed by the data processing unit51, and is further used as the storage area for various kinds of parameters.

The server30includes a data processing unit71, a communication unit72, and a storage unit73.

The data processing unit71of the server30performs various kinds of data processing to conduct a data distribution service. For example, the configuration data of the data distribution service is generated, and transmission thereof is controlled.

Specifically, a packet containing image and audio data is generated and distributed. Further, various kinds of notification information are generated and distributed.

The communication unit72transmits image, audio, and other data generated by the data processing unit71.

The storage unit73stores the data of images, sound, and the like to be distributed.

The storage unit73is also used as the work area for the data processing to be performed by the data processing unit71, and is further used as the storage area for various kinds of parameters.

FIG. 3is a diagram showing the structure of the data processing unit51of the client20in detail.

The data processing unit51of the client20includes a packet dividing unit81, an image data processing unit82, an audio data processing unit83, and a control unit84.

The packet dividing unit81divides a received packet into respective types of data (images, sound, and other control signals) based on header information about the packet received via the communication unit52, and supplies the respective divided packets to the respective data processing units.

The image data processing unit82acquires image data from a packet containing image data, and performs processes necessary for reproduction images, such as a decoding process.

The audio data processing unit83acquires audio data from a packet containing audio data, and performs processes necessary for reproduction of sound, such as a decoding process.

The control unit84controls the processes to be performed at the image data processing unit82and the audio data processing unit83.

3. Details of the Reproduction Speed Control Process to be Performed by the Audio Data Processing Unit

Next, the reproduction speed control process to be performed on audio data by the audio data processing unit83of the client20is described in detail.

FIG. 4is a diagram showing a specific example structure of the audio data processing unit83of the client20.

The audio data processing unit83includes an audio data storage memory101, a packet reception state monitoring unit102, a memory-stored data number monitoring unit103, and a reproduction control unit104.

The audio data storage memory101is a memory that stores packets received from the server30, and sequentially stores audio packets containing audio data selected by the packet dividing unit81.

P1, P2, . . . , Pn shown in the drawing represent the audio packets.

Each of these packets stores reproduction audio data of a predetermined length (several msec, for example).

The packet reception state monitoring unit102detects reception intervals of audio packets received from the server30.

The server30transmits audio packets at predetermined intervals. However, the reception intervals of audio packets received by the client20are not uniform due to a congestion state of the network or the like, and fluctuations or so-called jitter occurs.

The packet reception state monitoring unit102detects the reception intervals of respective packets containing audio data. In accordance with the detected packet reception intervals, the packet reception state monitoring unit102calculates a target stored data number (target buffer number)122in the audio data storage memory101, and outputs the target stored data number (target buffer number)122to the memory-stored data number monitoring unit103.

For example, if audio packets are received at regular intervals, and sound reproduction is performed at constant speed, the number of packets stored in the audio data storage memory101is substantially maintained constant.

However, if the network is congested, and packet reception is delayed, packet inputs to the memory101do not keep up with packet consumption accompanying a reproduction process, and the number of packets stored in the audio data storage memory101decreases. If the delay is long, the memory101eventually runs out of packets to be reproduced, and reproduced sound might become intermittent.

So as to prevent such a situation, the audio data processing unit83of the client20of the present disclosure changes the audio reproduction speed in accordance with packet reception states.

The packet reception state monitoring unit102calculates the target stored data number (target buffer number)122as the data to be used in controlling the audio reproduction speed, and outputs the target stored data number122to the memory-stored data number monitoring unit103.

The packet reception state monitoring unit102detects the reception intervals of packets containing audio data received by the communication unit52. In accordance with the detected packet reception intervals, the packet reception state monitoring unit102calculates the target stored data number (target buffer number)122in the audio data storage memory101, and outputs the target stored data number122to the memory-stored data number monitoring unit103.

The memory-stored data number monitoring unit103acquires the memory-stored data number (current buffer number)121calculated based on the number of data packets stored in the audio data storage memory101, and further receives the target stored data number (target buffer number)122from the packet reception state monitoring unit102.

The memory-stored data number monitoring unit103generates reproduction speed control information (speed control)123based on the following two pieces of data: (a) the memory-stored data number (current buffer number)121and (b) the target stored data number (target buffer number)122. The memory-stored data number monitoring unit103then outputs the reproduction speed control information (speed control)123to the reproduction control unit104.

The reproduction control unit104sequentially acquires audio packets from the audio data storage memory101, and performs processes necessary for reproduction, such as decoding. The results are output to the output unit105such as a speaker.

In this audio reproduction process, the reproduction control unit104performs a reproduction process while controlling the reproduction speed based on the reproduction speed control information (speed control)123that is input from the memory-stored data number monitoring unit103.

The audio data processing unit83of the client20of the present disclosure calculates a target number of packets to be stored in the audio data storage memory101, and changes the reproduction speed in accordance with a difference between the number of packets actually stored in the memory and the target number.

Referring now toFIG. 5, this reproduction speed control process is described in detail.

In the graph shown inFIG. 5, the abscissa axis indicates [(memory-stored data number)−(target stored data number)] (number of packets), and the ordinate axis indicates reproduction speed (%). The graph shows the correspondence relationship between the number of packets and the reproduction speed.

At the position of 0 on the abscissa axis, [(memory-stored data number)−(target stored data number)] is 0, and the number of the audio data packets stored in the memory101is equal to the target number.

At the position of +1, the memory-stored data number is larger than the target number by 1.

At the position of −1, the memory-stored data number is smaller than the target number by 1.

The reproduction speed at the position where [(memory-stored data number)−(target stored data number)] (number of packets) is 0 on the abscissa axis is 100%.

The reproduction speed of 100% is specified as the regular reproduction speed.

When [(memory-stored data number)−(target stored data number)] on the abscissa axis has a positive value, such as +1, +2, or +3, control is conducted so that reproduction is performed at a higher reproduction speed than the regular reproduction speed.

That is, when the memory-stored data number is larger than the target stored data number, reproduction control is conducted in a high-speed reproduction mode to increase the reproduction speed and speed up packet consumption in the memory.

In the graph shown inFIG. 5, the reproduction speed in the high-speed reproduction mode is increased to 100.5% and then to 101%, with the regular speed being 100%.

When [(memory-stored data number)−(target stored data number)] on the abscissa axis has a negative value, such as −1 or −2, control is conducted so that reproduction is performed at a lower reproduction speed than the regular reproduction speed.

That is, when the memory-stored data number is smaller than the target stored data number, reproduction control is conducted in a low-speed reproduction mode to lower the reproduction speed and slow down packet consumption in the memory.

In the graph shown inFIG. 5, the reproduction speed in the low-speed reproduction mode is controlled to decrease to 99%, with the regular speed being 100%.

As can be seen from the graph inFIG. 5, the process of changing the reproduction speed is to be performed slowly, with hysteresis being given to transitions of [(memory-stored data number)−(target stored data number)].

For example, in a case where a transition from position a where [(memory-stored data number)−(target stored data number)] is 0 to position b where [(memory-stored data number)−(target stored data number)] is +1 occurs inFIG. 5, the reproduction speed at position b is maintained at 100%, which is the reproduction speed at position a.

When the memory-stored data number increases further, and a transition from position b to position c where [(memory-stored data number)−(target stored data number) is +2 occurs, the reproduction speed is increased by 0.5%, and is set at a reproduction speed of 100.5% at position c.

As described above, when successive increases in [(memory-stored data number)−(target stored data number)] are confirmed, the reproduction speed is increased by a predetermined amount (0.5% in this example).

The memory-stored data number increases further, and a transition from position c to position d where [(memory-stored data number)−(target stored data number)] is +3 occurs.

At this transition position d, however, the reproduction speed is maintained at 100.5%, which is the same as the reproduction speed at position c.

When the memory-stored data number increases further, and a transition from position d to position e where [(memory-stored data number)−(target stored data number) is +4 occurs, the reproduction speed is further increased by 0.5%, and is set at a reproduction speed of 101% at position e.

As described above, the process of changing the reproduction speed is controlled slowly, with predetermined hysteresis being given to transitions of [(memory-stored data number)−(target stored data number)].

In the example shown in the drawing, the reproduction speed is increased when the value of change in [(memory-stored data number)−(target stored data number)] is +2.

The amount of change in the reproduction speed is 0.5% at a time. Specifically, the reproduction speed is changed to 100.5% and then to 101%, with the regular reproduction speed being 100%.

The upper limit value the reproduction speed is 101%. This is because a higher reproduction speed than that might cause viewers to have a feeling of strangeness.

The same applies to a case where the memory-stored data number decreases from position e shown inFIG. 5, and changes further from position f to position c to position g to position a. Every time −2 is detected as the amount of change in the difference between the memory-stored data number and the target number, the reproduction speed is decreased stepwise from 101% to 100.5% to 100%.

When [(memory-stored data number)−(target stored data number)] on the abscissa axis has a negative value such as −1 or −2, or when the memory-stored data number is smaller than the target stored data number, reproduction control is conducted in the low-speed reproduction mode to lower the reproduction speed and slow down packet consumption in the memory.

In the graph shown inFIG. 5, the reproduction speed in the low-speed reproduction mode is controlled to decrease to 99%, with the regular speed being 100%.

For example, in a case where a transition from position a where [(memory-stored data number)−(target stored data number)] is 0 to position h where [(memory-stored data number)−(target stored data number)] is −1 occurs, the reproduction speed at position h is maintained at 100%, which is the reproduction speed at position a.

When the memory-stored data number decreases further, and a transition from position h to position i where [(memory-stored data number)−(target stored data number) is −2 occurs, the reproduction speed is lowered by 1%, and is set at a reproduction speed of 99% at position i.

When [(memory-stored data number)−(target stored data number)] changes in the positive direction, the reproduction speed is increased by +0.5% in accordance with an amount of change of +2. When [(memory-stored data number)−(target stored data number)] changes in the negative direction, on the other hand, the reproduction speed is lowered by −1.0% in accordance with an amount of change of −2.

This is because, when [(memory-stored data number)−(target stored data number)] changes in the negative direction, there is a possibility of intermittent sound due to a packets shortage, and packet consumption is preferably minimized.

In the low-speed reproduction mode, the lower limit value of the reproduction speed is set at 99%, with the regular reproduction speed being 100%. This is because a lower reproduction speed than that might cause viewers to have a feeling of strangeness.

It should be noted that the reproduction speed control and the hysteresis settings illustrated inFIG. 5are merely examples, and various other settings are possible.

For example, optimum settings may be preferably set in accordance with the reproduction length of the audio data contained in one packet.

4. Specific Example of the Audio Reproduction Speed Control Process

Next, a specific example of a process of calculating parameters to be used in performing the audio reproduction speed control process described above with reference toFIGS. 4 and 5is described.

As can be seen from the above description, the audio data processing unit83of the client20of the present disclosure sets a target number of packets to be stored in the audio data storage memory101, and changes the reproduction speed in accordance with a difference between the number of packets actually stored in the memory and the target number.

One of the parameters necessary in the reproduction speed control is [(memory-stored data number)−(target stored data number)], which is indicated by the abscissa axis of the graph shown inFIG. 5.

As shown inFIG. 4, the memory-stored data number is the memory-stored data number (current buffer number)121calculated based on the data stored in the audio data storage memory101.

Also, as shown inFIG. 4, the target stored data number is the target stored data number (target buffer number)122calculated by the packet reception state monitoring unit102in accordance with packet reception intervals.

The memory-stored data number monitoring unit103generates the reproduction speed control information (speed control)123based on the two pieces of data, which are (a) the memory-stored data number (current buffer number)121and (b) the target stored data number (target buffer number)122. The memory-stored data number monitoring unit103then outputs the reproduction speed control information (speed control)123to the reproduction control unit104.

The reproduction control unit104sequentially acquires audio packets from the audio data storage memory101, and performs processes necessary for reproduction, such as decoding. The results are output to the output unit105such as a speaker.

In this audio reproduction process, the reproduction control unit104performs a reproduction process while controlling the reproduction speed based on the reproduction speed control information (speed control)123that is input from the memory-stored data number monitoring unit103.

This series of processes are performed on a regular basis.

The parameters to be used in this series of processes are the following three parameters: (a) the memory-stored data number (current buffer number)121, (b) the target stored data number (target buffer number)122, and (c) the reproduction speed control information (speed control)123.

The timings to update those three parameters (a) through (c) may be the same timings or may be different timings.

As an example, a process of updating (a) the memory-stored data number (current buffer number)121every two seconds, (b) the target stored data number (target buffer number)122every ten seconds, and (c) the reproduction speed control information (speed control)123every two seconds is described below.

First, an example of a process of calculating (a) the memory-stored data number (current buffer number)121is described.

The memory-stored data number monitoring unit103checks on the audio data storage memory101every time a new packet is input to the memory, and acquires the number of the packets stored in the memory.

In a case where a packet containing audio data is input approximately every 10 msec, for example, 200 checks are performed in two seconds.

The memory-stored data number monitoring unit103calculates the (a) memory-stored data number (current buffer number)121based on the average value (average_buffer_number) of approximately 200 pieces of data obtained through the approximately 200 memory checks performed in the two seconds.

Specifically, the memory-stored data number (current buffer number)121is calculated according to the following mathematical formula (1).
Memory-stored data number(current buffer number)=average_buffer_number=(total number of pieces of memory-stored data at time ofnth memory check)/n(1)

In the above formula (1), n represents the number of times the memory101is checked on at intervals equivalent to the output intervals of the reproduction speed control signal (speed control)123that is output from the memory-stored data number monitoring unit103.

In this example, the reproduction speed control signal (speed control)123is output from the memory-stored data number monitoring unit103every two seconds, and accordingly, n is equivalent to the number of times a check is performed in two seconds. In a case where a check is performed every time a packet is input every 10 msec, n is 200.

Next, an example of a process of calculating (b) the target stored data number (target buffer number)122is described.

The packet reception state monitoring unit102detects the reception intervals of packets containing audio data received by the communication unit52. In accordance with the detected packet reception intervals, the packet reception state monitoring unit102calculates the target stored data number (target buffer number)122in the audio data storage memory101, and outputs the target stored data number122to the memory-stored data number monitoring unit103.

The packet reception state monitoring unit102updates the target stored data number (target buffer number)122at predetermined intervals, and outputs the target stored data number122to the memory-stored data number monitoring unit103.

Here, an example case where the target stored data number (target buffer number)122is updated every ten seconds and is output to the memory-stored data number monitoring unit103is described.

The packet reception state monitoring unit102calculates the statistic variation in the input intervals of audio data packets input to the audio data processing unit83, generates the target stored data number (target buffer number)122based on the calculated value of the variation, and outputs the target stored data number (target buffer number)122to the reproduction control unit104.

An example where “standard deviation” is used as one of the index values of the static variation in the input intervals of audio data packets input to the audio data processing unit83is described below.

Specifically, the standard deviation of the input intervals of audio data packets is used as the index value indicating the variation in the packet input intervals.

As the variation in the packet input intervals becomes wider, the standard deviation of the packet input intervals becomes larger. As the variation in the packet input intervals becomes narrower, the standard deviation of the packet input intervals becomes smaller.

The packet reception state monitoring unit102first calculates the variance of the input intervals of audio data packets input to the audio data processing unit83, and calculates the standard deviation (stdDev=√(variance)) based on the calculated variance.

The packet reception state monitoring unit102performs the calculation described below every time an audio data packet is input to the audio data processing unit83, to update the variance of the input intervals of audio data packets input to the audio data processing unit83.

In the ten seconds as the period of setting the timing to update the target stored data number (target buffer number)122, a packet input interval is detected every time a packet is input, and the variance updating is continued based on the detected value.

Specifically, the packet input intervals of the packets that are input in the ten seconds are successively detected, and the variance of the packet input intervals is calculated.

The standard deviation (stdDev=√(variance)) of the packet input terminals is then calculated based on the calculated variance value.

Further, the target stored data number (target buffer number)122is calculated by using the calculated standard deviation (stdDev=√(variance)).

The variance calculation process to be performed by the packet reception state monitoring unit102is now described.

First, as an initial value setting process, the following parameters are set at 0.

In the above equations, n represents the number of times the variance value is updated (=the number of audio packet inputs), “mean” represents the mean value of the already measured packet intervals, and M2 represents the parameter applied to the variance calculation.

The following parameter updating is performed every time a packet is input.
n=n+1
delta=x−mean
mean=mean+delta/n
M2=M2+delta×(x−mean)
Variance=M2/(n−1)

In the above equations, x represents the packet interval between a newly input audio packet and the previously input audio packet, and “mean” represents the mean value of already measured packet intervals.

The packet reception state monitoring unit102performs the above described parameter updating process every time an audio data packet is input to the audio data processing unit83, and continues the parameter updating process for ten seconds, which is the period of setting the timing to update the target stored data number (target buffer number)122.

As the above described parameter updating is continued for ten seconds, the variance of the packet reception intervals of the packets received in the ten seconds is calculated.

After calculating the variance of the packet reception intervals of the packets received in the ten seconds from the start of the variance updating process, the packet reception state monitoring unit102calculates the standard deviation (stdDev) of the packet reception intervals according to the mathematical formula (2) shown below based on the calculated variance.
Standard deviation(stdDev)=sqrt(variance)   (2)

In the above formula (2), sqrt represents a √ calculation (a square-root calculation).

Further, the packet reception state monitoring unit102calculates the target stored data number (target buffer number)122according to the mathematical formula (3) shown below by using the standard deviation (stdDev) of the packet reception intervals calculated according to the above mathematical formula (2) in the ten seconds.
Target stored data number=3+floor(stdDev)  (3)

In the above mathematical formula (3), “floor(stdDev)” represents the floor function for truncating the standard deviation (stdDev) to an integer.

The target stored data number (target buffer number)122calculated according to the above mathematical formula (3) may have any value, but an allowable minimum value and an allowable maximum value may be determined in advance.

For example, a minimum value (min) of 4 and a maximum value (max) of 8 may be set as an allowable range so that the target stored data number is set at 4 when the value calculated according to the above mathematical formula (3) is 3 or smaller, and the target stored data number is set at 8 when the value calculated according to the above mathematical formula (3) is 9 or greater.

This allowable range is merely an example, and an optimum range is preferably set in accordance with the largest possible number of pieces of data to be stored in the memory and the amount of data contained in each packet.

The packet reception state monitoring unit102calculates the target stored data number (target buffer number)122according to the above mathematical formula (3) on a regular basis, such as every ten seconds, and outputs the target stored data number (target buffer number)122to the memory-stored data number monitoring unit103.

To sum up those processes, the memory-stored data number monitoring unit103performs the following processes.

(Process 2) Receiving updated data of the target stored data number (target buffer number)122from the packet reception state monitoring unit102every ten seconds.

(Process 3) Calculating [(memory-stored data number)−(target stored data number)] every two seconds, which is the update interval of the memory-stored data number (current buffer number)121.

This calculated value corresponds to the abscissa axis of the graph shown inFIG. 5.

(Process 4) Generating the reproduction speed control information (speed control)123based on the value of [(memory-stored data number)−(target stored data number)] calculated every two second, and outputting the reproduction speed control information (speed control)123to the reproduction control unit104.

The processes to be performed by the memory-stored data number monitoring unit103are the above processes.

In (process 4), the reproduction speed control information (speed control)123to be generated based on [(memory-stored data number)−(target stored data number)] is set as follows.

In a case where [(memory-stored data number)−(target stored data number)] is 0, or where the memory-stored data number is equal to the target stored data number, the reproduction speed control information (speed control)123is generated so as to maintain the current reproduction speed, and is output to the reproduction control unit104.

This process corresponds to point a in the graph shown inFIG. 5.

In a case where [(memory-stored data number)−(target stored data number)] is greater than 0, or where the memory-stored data number is larger than the target stored data number, reproduction control is performed in the high-speed reproduction mode. However, the reproduction speed changing process is performed, with hysteresis being given to transitions of the difference between the target number and the number of actually stored packets.

This process corresponds to points a through g in the graph shown inFIG. 5.

The reproduction control in the high-speed reproduction mode is performed in a case where [(memory-stored data number)−(target stored data number)] is greater than 0.

As described above with reference toFIG. 5, the reproduction speed changing process is performed, with hysteresis being given to transitions of the difference between the target stored data number and the memory-stored data number.

For example, in the specific example described with reference toFIG. 5, the reproduction speed is increased when two successive increases in [(memory-stored data number)−(target stored data number)] are detected. The reproduction speed is lowered when two successive decreases in [(memory-stored data number)−(target stored data number)] are detected.

However, an upper limit value of the reproduction speed is set, so that reproduction at a higher speed than the upper speed value is not performed. In the example illustrated inFIG. 5, the upper limit value of the reproduction speed is 101% of the regular reproduction speed.

In a case where [(memory-stored data number)−(target stored data number)] is smaller than 0, or where the memory-stored data number is smaller than the target stored data number, reproduction control is performed in the low-speed reproduction mode. However, the reproduction speed is changed, with hysteresis being given to transitions of the difference between the target number and the number of actually stored packets. This process corresponds to point a and points h through j in the graph shown inFIG. 5.

The reproduction control in the low-speed reproduction mode is performed in a case where [(memory-stored data number)−(target stored data number)] is smaller than 0.

As described above with reference toFIG. 5, the reproduction speed is changed, with hysteresis being given to transitions of the difference between the target stored data number and the memory-stored data number.

For example, in the specific example described with reference toFIG. 5, the reproduction speed is lowered when two successive decreases in [(memory-stored data number)−(target stored data number)] are detected. The reproduction speed is increased when two successive increases in [(memory-stored data number)−(target stored data number)] are detected.

However, a lower limit value of the reproduction speed is set, so that reproduction at a lower speed than the lower speed value is not performed. In the example illustrated inFIG. 5, the lower limit value of the reproduction speed is 99% of the regular reproduction speed.

As described above, the memory-stored data number monitoring unit103in the above described (process 4) generates the reproduction speed control information (speed control)123based on [(memory-stored data number)−(target stored data number)], and outputs the reproduction speed control information (speed control)123to the reproduction control unit104.

Since the memory-stored data number is updated every two seconds, [(memory-stored data number)−(target stored data number)] is also updated every two seconds in the above described example.

The memory-stored data number monitoring unit103in the above described (process 4) generates the reproduction speed control information (speed control)123every two seconds based on [(memory-stored data number)−(target stored data number)], which is updated every two seconds, and outputs the reproduction speed control information (speed control)123to the reproduction control unit104.

As described above with reference toFIG. 5, the reproduction speed changing process is performed, with hysteresis being given to transitions of [(memory-stored data number)−(target stored data number)].

In the example illustrated inFIG. 5, the reproduction speed control information (speed control)123to be output to the reproduction control unit104serves as the control information for performing reproduction at one of the following four reproduction speeds.

In the above described (process 4), the memory-stored data number monitoring unit103determines at which one of points a through i in the transition diagram shown inFIG. 5[(memory-stored data number)−(target stored data number)] being updated every two seconds is located.

The memory-stored data number monitoring unit103acquires the reproduction speed (one of the above (1) through (4)) corresponding to the determined location, generates the reproduction speed control information (speed control)123for performing reproduction at the acquired reproduction speed, and outputs the reproduction speed control information (speed control)123to the reproduction control unit104.

5. Sequence of the Reproduction Speed Control Process to be Performed by the Audio Data Processing Unit

Referring now to the flowchart shown inFIG. 6, the reproduction speed control sequence to be performed by the audio data processing unit83is described. The flow shown inFIG. 6is realized by performing a process in accordance with a program stored in the storage unit under the control of the control unit of the client20, for example.

In the following, the procedures of the respective steps in the flowchart shown inFIG. 6are sequentially described.

First, the audio data processing unit83detects the variation in the packet reception intervals of received audio packets.

This process is a process to be performed by the packet reception state monitoring unit102.

The packet reception state monitoring unit102detects reception intervals of packets containing audio data received from the server30.

The server30transmits audio packets at predetermined intervals. However, the reception intervals of audio packets received by the client20are not uniform due to a congestion state of the network or the like, and fluctuations or so-called jitter occurs.

The packet reception state monitoring unit102calculates the variation in the reception intervals of the respective packets containing audio data.

As described above, the standard deviation of reception intervals of packets containing audio data can be used as an index value of variation, for example.

When the standard deviation is calculated, the packet reception state monitoring unit102first calculates the variance of the input intervals of audio data packets input to the audio data processing unit83, and calculates the standard deviation (stdDev=√(variance)) based on the calculated variance.

The index value of variation is not limited to the standard deviation, and some other value may be used. For example, the value of a longer reception interval than the mean value of packet reception intervals may be used.

The audio data processing unit83calculates the target stored data number in the memory based on the detected variation. A large target number is set when the variation is wide, and a small target number is set when the variation is narrow.

This process is also a process to be performed by the packet reception state monitoring unit102. The packet reception state monitoring unit102detects the reception intervals of packets containing audio data received by the communication unit52. In accordance with the detected packet reception intervals, the packet reception state monitoring unit102calculates the target stored data number (target buffer number)122in the audio data storage memory101, and outputs the target stored data number122to the memory-stored data number monitoring unit103.

In a case where the standard deviation (stdDev=√(variance)) is calculated as the variation in packet reception intervals in step S101, for example, the target stored data number122can be calculated according to the above described mathematical formula (3).

That is, the target stored data number (target buffer number)122is expressed as
“target stored data number”=3+floor(stdDev)  (3).

The target stored data number (target buffer number)122is calculated according to the above described mathematical formula (3), and is output to the memory-stored data number monitoring unit103.

In step S103, the audio data processing unit83calculates the difference between the target stored data number and the actual memory-stored data number. This process is a process to be performed by the memory-stored data number monitoring unit103.

The memory-stored data number monitoring unit103detects the memory-stored data number (current buffer number)121on a regular basis (every two seconds, for example), and calculates the difference from the target stored data number (target buffer number)122that is input from the packet reception state monitoring unit102.

That is, [(memory-stored data number)−(target stored data number)] is calculated on a regular basis. This calculated value corresponds to the abscissa axis of the graph shown inFIG. 5.

The memory-stored data number monitoring unit103determines whether the memory-stored data number is smaller than the target stored data number, or is larger than the target stored data number.

Specifically, a check is made to determine whether the difference value of [(memory-stored data number)−(target stored data number)] calculated in step S103is a positive value, is a negative value, or is 0.

If the memory-stored data number is equal to the target stored data number, the process moves on to step S105.

If the memory-stored data number is smaller than the target stored data number, the process moves on to step S106.

If the memory-stored data number is larger than the target stored data number, the process moves on to step S107.

The processes in steps S105through S107are processes to be performed by the reproduction control unit104based on the reproduction speed control information (speed control) that is output from the memory-stored data number monitoring unit103.

If the memory-stored data number is determined to be equal to the target stored data number in the determination process in step S104, the process moves on to step S105.

In step S105, reproduction speed control is performed so as to maintain the current reproduction speed.

This is a process to be performed based on the reproduction speed control information (speed control) that is output from the memory-stored data number monitoring unit103.

In the case where [(memory-stored data number)−(target stored data number)] is 0, the memory-stored data number monitoring unit103generates the reproduction speed control information (speed control) so as to maintain the current reproduction speed, and outputs the reproduction speed control information (speed control) to the reproduction control unit104.

Based on this reproduction speed control information (speed control), the reproduction control unit104performs reproduction speed control so as to maintain the current reproduction speed.

This process corresponds to point a in the graph shown inFIG. 5.

If the memory-stored data number is determined to be smaller than the target stored data number in the determination process in step S104, the process moves on to step S106.

In step S106, reproduction control is performed in the low-speed reproduction mode. However, the reproduction speed is changed, with hysteresis being given to transitions of the difference between the target number and the number of actually stored packets.

This is a process to be performed by the reproduction control unit104based on the reproduction speed control information (speed control) that is output from the memory-stored data number monitoring unit103.

This process corresponds to points h through in the graph shown inFIG. 5.

As described above with reference toFIG. 5, the reproduction speed is changed, with hysteresis being given to transitions of the difference between the target stored data number and the memory-stored data number.

For example, in the specific example described with reference toFIG. 5, the reproduction speed is lowered when two successive decreases in [(memory-stored data number)−(target stored data number)] are detected. The reproduction speed is increased when two successive increases in [(memory-stored data number)−(target stored data number)] are detected.

However, a lower limit value of the reproduction speed is set, so that reproduction at a lower speed than the lower speed value is not performed. In the example illustrated inFIG. 5, the lower limit value of the reproduction speed is 99% of the regular reproduction speed.

If the memory-stored data number is determined to be larger than the target stored data number in the determination process in step S104, the process moves on to step S107.

In step S107, reproduction control is performed in the high-speed reproduction mode. However, the reproduction speed is changed, with hysteresis being given to transitions of the difference between the target number and the number of actually stored packets.

This is a process to be performed by the reproduction control unit104based on the reproduction speed control information (speed control) that is output from the memory-stored data number monitoring unit103.

This process corresponds to points b through g in the graph shown inFIG. 5.

The reproduction control in the high-speed reproduction mode is performed in a case where [(memory-stored data number)−(target stored data number)] is greater than 0.

For example, in the specific example described with reference toFIG. 5, the reproduction speed is increased when two successive increases in [(memory-stored data number)−(target stored data number)] are detected. The reproduction speed is lowered when two successive decreases in [(memory-stored data number)−(target stored data number)] are detected.

However, an upper limit value of the reproduction speed is set, so that reproduction at a higher speed than the upper speed value is not performed. In the example illustrated inFIG. 5, the upper limit value of the reproduction speed is 101% of the regular reproduction speed.

In this manner, audio reproduction speed control is performed.

In the present disclosure, the memory-stored data number is dynamically changed in accordance with packet reception intervals, and the reproduction speed changing process is performed in accordance with the difference between the target value and the number of actually stored packets.

Also, the reproduction speed changing process is performed, with hysteresis being given to transitions of the difference between the target number and the number of actually stored packets.

As such a process is performed, natural sound reproduction that will not cause viewers to have a feeling of strangeness can be realized, and the possibility of intermittent sound generation due to a shortage of packets stored in the memory can be reduced.

6. Example of an Image Data Reproduction Control Process to be Performed by the Image Data Processing Unit

Next, an image reproduction control process to be performed in the client20is described.

FIG. 7is a diagram showing an example structure of the image data processing unit82of the client20.

As shown inFIG. 7, the image data processing unit82includes an image data storage memory201, a packet reception state monitoring unit202, a phase offset monitoring unit203, and a reproduction control unit204. The reproduction control unit204outputs images to the output unit205formed with a display such as an LCD. The reproduction control unit204and the output unit205such as a display are connected by an HDMI cable, for example.

The image data storage memory201is a memory that stores packets received from the server30, and sequentially stores image packets containing image data selected by the packet dividing unit81.

P1, P2, . . . , Pn shown in the drawing represent the image packets.

Each of those packets contains reproduction image data.

The packet reception state monitoring unit202detects reception intervals of packets containing image data received from the server30.

The server30transmits image packets at predetermined intervals. However, the reception intervals of image packets received by the client20are not uniform due to a congestion state of the network or the like, and fluctuations or so-called jitter occurs.

The packet reception state monitoring unit202detects the reception intervals of respective packets containing image data. In accordance with the detected packet reception intervals, the packet reception state monitoring unit202calculates a target phase offset222, and outputs the target phase offset222to the phase offset monitoring unit203.

Here, a phase offset is a difference between a vertical synchronization signal (Vsync) of the server30and a vertical synchronization signal (Vsync) of the client20. Referring now toFIG. 8, a phase offset is described.

FIG. 8is a diagram showing example sequences of image data transmission and reception in a case where delays are short.

InFIG. 8, the following two chronological sequences are shown in time lines (t).

(A) Server transmission image frame

(B) Client reproduction image frame

The numerical values “1”, “2”, “3”, and “4” in (A) and (B) are image frame numbers. Specifically, “1” represents image frame1, and “2” represents image frame2. Each of these frames is an image frame forming a moving image (video).

InFIG. 8, (A) shows that respective image frames1,2,3,4, . . . are transmitted at respective timings (t1(s), t2(s), t3(s), t4(s), . . . ) to output a vertical synchronization signal (Vsync) in accordance with the server clock of the server30.

The transmission image frames1,2,3, and4are received by the client20via a network.

The timings to receive the image frames1,2,3,4, . . . at the client20are indicated by the edges of the arrows of the respective frames1,2,3,4, . . . of (A) server transmission image frame. Those are the locations of the edges of the arrows shown on the time line (t) indicating Vsync of the client20.

Based on the vertical synchronization signal (Vsync) generated in accordance with the client clock, the client20outputs the image frames1,2,3, and4received from the server30to the display unit.

The image frames1,2,3, and4are then sequentially reproduced as shown in (B) client reproduction image frame.

In the case of a moving image of 60 fps, for example, any of the display intervals of the image frames1,2,3, and4is 1/60 sec.

The respective image frames1,2,3, and4are displayed at the timings ((t1(c), t2(c), t3(c), and t4(c)) to generate the vertical synchronization signal (Vsync) shown in the time line of Vsync of the client20.

A phase offset is a difference between the vertical synchronization signal (Vsync) of the server30and the vertical synchronization signal (Vsync) of the client20.

In the example illustrated inFIG. 8, the difference between t1(c) and t1(s) is equivalent to a phase difference offset.

The packet reception state monitoring unit202of the image data processing unit82shown inFIG. 7detects the reception intervals of respective packets containing image data. In accordance with the detected packet reception intervals, the packet reception state monitoring unit202calculates the target phase offset222, and outputs the target phase offset222to the phase offset monitoring unit203.

The target phase offset222is made larger when the variation in the reception intervals of respective packets containing image data is wide, and is made smaller when the variation in the reception intervals of respective packets containing image data is narrow. The packet reception state monitoring unit202outputs the target phase offset222, which is set in this manner, to the phase offset monitoring unit203.

The phase offset monitoring unit203acquires the actual phase offset (current phase offset)221of the image packet currently stored in the image data storage memory201.

This actual phase offset221is acquired from temporal information about the vertical synchronization signal (Server vsync) of the server30. The temporal information is set as metadata of received image frames.

The phase offset monitoring unit203generates phase control information (phase control)223based on the two kinds of data: (a) the actual phase offset (current phase offset)221, and (b) the target phase offset222. The phase control information (phase control)223is then output to the reproduction control unit204.

The reproduction control unit204sequentially acquires image packets from the image data storage memory201, and performs processes such as decoding. The results are output to the output unit205such as a display.

In this image reproduction process, the reproduction control unit204performs an image reproduction process with an optimum phase offset that is set based on the phase control information (phase control)223, which is input from the phase offset monitoring unit203.

Specifically, when the variation in packet reception intervals is narrow (or the jitter is low), the phase offset is made smaller.

When the variation in packet reception intervals is wide (or the jitter is high), the phase offset is made larger.

The image reproduction process is performed by controlling the phase offset in such a manner.

The example illustrated inFIG. 8is an example process to be performed when the variation in packet reception intervals is narrow (or the jitter is low).

As described above, when the variation in packet reception intervals is narrow (or the jitter is low), image reproduction is performed with a small phase offset. As a small phase offset is set, reproduction with shorter reproduction delays can be realized.

The example illustrated inFIG. 9is an example process to be performed when the variation in packet reception intervals is wide (or the jitter is high).

When the variation in packet reception intervals is wide (or the jitter is high), image reproduction is performed with a large phase offset. As a large phase offset is set, reproduction delays become longer. However, even when there is a long delay, or when the period from transmission from the server to reception at the client is long as in the case of image frame4, for example, image frame4can be output at the timing (t5(c)) to output the synchronization signal (Vsync) of the client without delay. As a result, smooth image reproduction without generation of a discontinuous image can be performed.

This image reproduction process is performed by taking into account both the vertical synchronization signal (Server vsync) based on the server clock of the server30and the vertical synchronization signal (Client vsync) based on the client clock of the client20. Accordingly, an effect to correct a clock shift between the server30and the client20is also achieved.

7. Sequence of the Reproduction Control Process to be Performed by the Image Data Processing Unit

Referring now to the flowchart shown inFIG. 10, the reproduction control sequence to be performed by the image data processing unit82is described.

The flow shown inFIG. 10is realized by performing a process in accordance with a program stored in the storage unit under the control of the control unit of the client20, for example.

In the following, the procedures of the respective steps in the flowchart shown inFIG. 10are sequentially described.

First, the image data processing unit82detects the variation in the packet reception intervals of received image packets.

This process is a process to be performed by the packet reception state monitoring unit202.

The packet reception state monitoring unit202detects reception intervals of packets containing image data received from the server30.

The server30transmits image packets at predetermined intervals. However, the reception intervals of image packets received by the client20are not uniform due to a congestion state of the network or the like, and fluctuations or so-called jitter occurs.

The packet reception state monitoring unit202calculates the variation in the reception intervals of the respective packets containing image data.

The standard deviation of reception intervals of packets containing image data can be used as an index value of variation, for example.

When the standard deviation is calculated, the packet reception state monitoring unit202first calculates the variance of the input intervals of image data packets input to the image data processing unit82, and calculates the standard deviation (stdDev=√(variance)) based on the calculated variance.

The index value of variation is not limited to the standard deviation, and some other value may be used. For example, the value of a longer reception interval than the mean value of packet reception intervals may be used.

The image data processing unit82calculates a target phase offset based on the detected variation. This process is also a process to be performed by the packet reception state monitoring unit202. The packet reception state monitoring unit202calculates the target phase offset222based on the variation in the packets containing image data received by the communication unit52, and outputs the target phase offset222to the phase offset monitoring unit203.

The target phase offset222is made larger when the variation in the reception intervals of respective packets containing image data is wide, and is made smaller when the variation in the reception intervals of respective packets containing image data is narrow.

The packet reception state monitoring unit202outputs the target phase offset222, which is set in this manner, to the phase offset monitoring unit203.

The processes in steps S203through S205are processes to be performed by the phase offset monitoring unit203and the reproduction control unit204.

The phase offset monitoring unit203acquires the actual phase offset (current phase offset)221of the image packet currently stored in the image data storage memory201.

This actual phase offset221is acquired from temporal information about the vertical synchronization signal (Server vsync) of the server30. The temporal information is set as metadata of received image frames.

The phase offset monitoring unit203generates the phase control information (phase control)223based on the two kinds of data: (a) the actual phase offset (current phase offset)221, and (b) the target phase offset222. The phase control information (phase control)223is then output to the reproduction control unit204.

The reproduction control unit204sequentially acquires image packets from the image data storage memory201, and performs processes such as decoding. The results are output to the output unit205such as a display.

In this image reproduction process, the reproduction control unit204performs an image reproduction process with an optimum phase offset that is set based on the phase control information (phase control)223, which is input from the phase offset monitoring unit203.

Specifically, when the variation in packet reception intervals is narrow (or the jitter is low), the process moves from step S203on to step S204, and reproduction control is performed with a smaller phase offset (a smaller difference between Vsync of the server and Vsync of the client).

This corresponds to the example process described above with reference toFIG. 8.

When the variation in the packet reception intervals is wide (or the jitter is high), on the other hand, the process moves from step S203on to step S205, and reproduction control is performed with a larger phase offset (a larger difference between Vsync of the server and Vsync of the client).

This corresponds to the example process described above with reference toFIG. 9.

The image reproduction process is performed by controlling the phase offset in such a manner.

By performing the above described image reproduction, a small phase offset is set when the jitter is low. Accordingly, a reproduction process that causes only short reproduction delays is realized.

When the jitter is high, on the other hand, a large phase offset is set. Accordingly, even when a packet is received after a long delay, smooth reproduction without generation of a discontinuous image is realized.

8. Example Hardware Configuration of the Client

Referring now toFIG. 11, an example configuration of a communication device serving as the client is described.FIG. 11shows an example hardware configuration of the client.

A CPU (Central Processing Unit)401functions as a data processing unit that performs various kinds of processes in accordance with programs stored in a ROM (Read Only Memory)402or a storage unit408. For example, processes in accordance with the sequences described in the above embodiment are carried out. Programs to be executed by the CPU401and data are stored in a RAM (Random Access Memory)403. The CPU401, the ROM402, and the RAM403are connected to one another by a bus404.

The CPU401is connected to an input/output interface405via the bus404. Further, an input unit406formed with various switches, a keyboard, a mouse, a microphone, and the like, and an output unit407formed with a display, a speaker, and the like are connected to the input/output interface405. The CPU401performs various kinds of processes in accordance with instructions input through the input unit406, and outputs results of the processes to the output unit407, for example.

The storage unit408connected to the input/output interface405is formed with a hard disk, for example, and stores programs to be executed by the CPU401and various kinds of data. A communication unit409functions as a transmission/reception unit for data communications through communication channels such as the Internet, a local area network, and broadcast waves, and communicates with external devices.

A drive410connected to the input/output interface405drives a removable medium411that is a magnetic disk, an optical disk, a magnetooptical disk, or a semiconductor memory such as a memory card, and performs data recording or reading.

Data encoding and decoding can be performed as a process by the CPU401serving as a data processing unit. However, a codec as special-purpose hardware for performing encoding processes and decoding processes may also be included.

9. Summary of the Structures of the Present Disclosure

Embodiments of the present disclosure have been described in detail with reference to specific examples. However, it is obvious that a person skilled in the art can make modifications to the embodiments or substitutions of the embodiments without departing from the scope of the present disclosure. That is, the present disclosure has been described in the form of embodiments, and those embodiments are not to be interpreted in a restrictive manner. The claims should be taken into account in understanding the subject matter of the present disclosure.

The present technique disclosed in this specification may also be embodied in the structures described below.

(1) A communication device including:

a packet reception state monitoring unit that monitors reception intervals of packets received by a communication unit, and generates and outputs a target value as a parameter for a reproduction control process based on the variation in the packet reception intervals; and
a reproduction control unit that performs reproduction control in accordance with the target value generated by the packet reception state monitoring unit,
wherein the reproduction control unit controls an audio reproduction speed or image frame reproduction in accordance with the target value.
(2) The communication device of (1),
wherein the packet reception state monitoring unit calculates a target stored data number as a target number of audio packets stored in a memory based on the variation in the reception intervals of audio packets received by the communication unit, and
the reproduction control unit controls the audio reproduction speed in accordance with the difference between the target stored data number and a memory-stored data number that is the number of audio packets actually stored in the memory.
(3) The communication device of (2), wherein the packet reception state monitoring unit makes the target stored data number larger when the variation in the reception intervals of the audio packets received by the communication unit is wide, and makes the target stored data number smaller when the variation is narrow.
(4) The communication device of (2) or (3), wherein the reproduction control unit performs reproduction control in a low-speed reproduction mode at a lower speed than a regular reproduction speed when the memory-stored data number is smaller than the target stored data number, and performs reproduction control in a high-speed reproduction mode at a higher speed than the regular reproduction speed when the memory-stored data number is larger than the target stored data number.
(5) The communication device of any of (2) through (4), wherein the reproduction control unit changes the audio reproduction speed by giving hysteresis to a transition of the difference between the memory-stored data number and the target stored data number.
(6) The communication device of any of (2) through (5), wherein the reproduction control unit performs an audio reproduction speed control process to increase the audio reproduction speed when the difference value between the memory-stored data number and the target stored data number continuously increases, and
reduce the audio reproduction speed when the difference value between the memory-stored data number and the target stored data number continuously decreases.
(7) The communication device of any of (2) through (6), wherein the packet reception state monitoring unit calculates the standard deviation of the reception intervals of audio packets received by the communication unit, to calculate the variation in the reception intervals of the audio packets.
(8) The communication device of (7), wherein the packet reception state monitoring unit makes the target stored data number larger when the standard deviation in the reception intervals of the audio packets is large, and makes the target stored data number smaller when the standard deviation in the reception intervals is small.
(9) The communication device of any of (2) through (8), wherein the memory-stored data number is the average value of the numbers of packets stored in the memory in a predetermined period of time.
(10) The communication device of (1),
wherein the packet reception state monitoring unit calculates a target phase offset as a target value of a phase offset equivalent to the difference between a server vertical synchronization signal of a server transmitting packets and a client vertical synchronization signal of a client that is the communication device based on the variation in the reception intervals of image packets received by the communication unit, and
the reproduction control unit performs image reproduction control by determining the timing to reproduce an image frame in accordance with the target phase offset.
(11) The communication device of (10), wherein the packet reception state monitoring unit makes the target phase offset larger when the variation in the reception intervals of the image packets received by the communication unit is wide, and makes the target phase offset smaller when the variation is narrow.
(12) The communication device of (10) or (11), wherein, when the variation in the reception intervals of the image packets is narrow, and the target phase offset is small, the reproduction control unit sets a small phase offset and performs image reproduction with shorter reproduction delays.
(13) The communication device of any of (10) through (12), wherein, when the variation in the reception intervals of the image packets is wide, and the target phase offset is large, the reproduction control unit sets a large phase offset and performs image reproduction with a reduced possibility of intermittent reproduction due to a packet delay.
(14) A data processing method implemented in a communication device, including:
a packet reception state monitoring unit monitoring reception intervals of packets received by a communication unit, and generating and outputting a target value as a parameter for a reproduction control process based on the variation in the packet reception intervals; and
a reproduction control unit performing reproduction control in accordance with the target value generated by the packet reception state monitoring unit,
wherein an audio reproduction speed or image frame reproduction is controlled in accordance with the target value in the reproduction control.

The series of processes described in this specification can be performed by hardware, software, or a combination of hardware and software. In a case where the processes are performed by software, a program in which the processing sequences are recorded is installed in a memory incorporated into special-purpose hardware in a computer, or the program is installed in a general-purpose computer that can perform various kinds of processes. Programs can be recorded beforehand in a recording medium, for example. Programs can be installed from the recording medium into a computer. Alternatively, programs may be received via a network such as the Internet, and be installed into a recording medium such as an internal hard disk.

The various processes described in this specification are carried out not only in chronological order in accordance with the disclosure, but also in parallel or independently of one another depending on the processing capacity of the device carrying out the processes or depending on the degree of necessity. In this specification, a “system” means a logical assembly of devices, and the respective devices are not necessarily located in the same housing.

INDUSTRIAL APPLICABILITY

As described above, an embodiment of the present disclosure realizes a device and a method for monitoring the variation in packet reception intervals and performing optimum reproduction control in accordance with the variation.

Specifically, a packet reception state monitoring unit calculates a target stored data number as a target number of audio packets stored in a memory based on the variation in the reception intervals of audio packets, and a reproduction control unit controls the audio reproduction speed in accordance with the difference between the target stored data number and a memory-stored data number. When the memory-stored data number is smaller than the target stored data number, reproduction control is performed in a low-speed reproduction mode at a lower speed than a regular reproduction speed. When the memory-stored data number is larger than the target stored data number, reproduction control is performed in a high-speed reproduction mode at a higher speed than the regular reproduction speed. A reproduction speed changing process is performed, with hysteresis being given to transitions of the difference between the memory-stored data number and the target stored data number.

With this structure, a device and a method for monitoring the variation in packet reception intervals and performing optimum reproduction control in accordance with the variation is realized.

REFERENCE SIGNS LIST