Source: https://patents.google.com/patent/TWI549436B/en
Timestamp: 2019-10-18 09:41:44
Document Index: 539403157

Matched Legal Cases: ['art 626', 'art 626', 'art 626', 'art 626', 'art 1000', 'art 1000']

TWI549436B - The method for inserting data block and equipment - Google Patents
The method for inserting data block and equipment Download PDF
TWI549436B
TWI549436B TW098136942A TW98136942A TWI549436B TW I549436 B TWI549436 B TW I549436B TW 098136942 A TW098136942 A TW 098136942A TW 98136942 A TW98136942 A TW 98136942A TW I549436 B TWI549436 B TW I549436B
TW098136942A
TW201023531A (en
2009-10-30 Application filed by Nokia Technologies Oy filed Critical Nokia Technologies Oy
2010-06-16 Publication of TW201023531A publication Critical patent/TW201023531A/en
2016-09-11 Publication of TWI549436B publication Critical patent/TWI549436B/en
Method and apparatus for inserting data blocks
This application is a summary of multimedia data streaming. In particular, the present application relates to inserting a multimedia data unit or a data portion in a data block.
This paragraph is intended to provide a background or context for the invention as recited in the scope of the claims. The description herein may include concepts that can be pursued, but need not be previously conceived or pursued. Therefore, the method described in this paragraph is not the prior art of the description and the scope of the patent application, and is not admitted to be prior art in this paragraph, unless otherwise indicated herein. .
The data transfer method can be divided into unicast, multicast, and broadcast according to the number of recipients and their user relationships according to the routing method used. Unicast refers to the transmission of media content to a user. In unicast, the recipient typically requests the transmission of the media content from the sender. Multicast is defined as the transmission of media content to a known group of users. In multicast, a user device, such as a communication device, typically combines a multicast session via a particular communication protocol. The broadcast service delivers media content to an unknown and possibly indiscriminate group of users. In the broadcast, a user device, such as a communication device, basically begins receiving the broadcast transmission without any notification to the sender.
Streaming is a collective term for streaming, streaming, and streaming applications. Streaming delivery is also used for applications other than streaming, such as video telephony. Streaming transfer is characterized by simultaneous reception and playback of the received data. Streaming delivery, such as the User Datagram Protocol (UDP), is often used with unreliable communication protocols to have a reasonable and relatively stable endpoint-to-end transmission delay. Progressive downloading means transmitting a multimedia file, such as the Hypertext Transfer Protocol (HTTP), over a reliable protocol and decoding and presenting the file while it is receiving a file.
Streaming services are becoming more common in network access technologies, such as the ability to provide sufficient bandwidth, and significant improvements in media coding techniques, such as improved quality. In streaming, the content plays immediately after the remote server begins receiving. The playback delay may range from a few seconds (eg, under the client-server architecture) up to one or two minutes (eg, point-to-point streaming applications).
Streaming is deployed in several different applications, such as on-demand video, Internet Protocol Television (IPTV), mobile TV broadcast/multicast, peer-to-peer streaming, and/or the like. Depending on the target application, different protocols can be used to set and control a stream of sessions. The 3rd Generation Partnership Project (3GPP) defines a unicast streaming service, which is a Packet-switched Streaming Service (PSS), which can store content strings on a wireless unicast carrier. Flow to mobile users. Digital Video Broadcast (DVB) defines an IPTV service that delivers live and stored content to a user's home on a fixed carrier (eg, a DSL line). This delivery can be performed in unicast or multicast mode.
The streaming service can utilize the Multimedia Broadcast/Multicast Service (MBMS) defined by 3GPP. MBMS uses a Forward Error Correction (FEC) mechanism to protect media streams from transmission errors and packet loss. The MBMS FEC code is a systematic block code that can generate a set of patch data from the original source material. The source material is separated into source blocks. The source block contains a set of source symbols, such as data packets or data units. A source block is then supplied to the FEC encoder to generate a patch symbol.
The FEC decoding operation is initiated at a communication device to recover from packet loss that occurred during the transmission. To this end, the communication device buffers the incoming packet of the current source block and the associated patching material. FEC decoding can begin each time the source block is received and the associated patch material is completed.
In multicast and broadcast streaming, scalable media coding can be used to provide different resolution levels for different receivers. For video streaming, it provides spatial, temporal, and quality scalability. Scalable Video Coding (SVC) has been defined as an extension of the Advanced Video Coding (H.264/AVC) standard. SVC allows for spatial scalability of spatial, temporal, coarse granularity and media granularity. Temporal scalability is achieved using architectural B-pictures, which have made it possible to utilize H.264/AVC codecs. Spatial scalability allows for the generation of two or more subsets of the bitstream, with each subset resulting in different image resolution. Quality scalability results in a bitstream layer that improves the signal-to-noise ratio (SNR) of the video signal. SVC requires a single loop decoder so that motion compensation can be performed at a single layer. Audio scalability can be of different types. SNR scalability represents a procedure in which the enhancement layer improves the SNR of the audio signal. Bandwidth expandability is an extended version of the audio layer that the extended layer can represent. Channel scalability represents the mode in which the expansion layer provides additional audio channels.
In one aspect, a method includes receiving a multimedia data block, the multimedia data block including a plurality of data portions, and inserting the plurality of data portions to form an inserted data block based on a priority order, the priority order being a multimedia material type based on the data portion; and transmitting the inserted data block.
In some examples, the method further includes generating an insertion attribute based on information associated with the inserted data block.
In other examples, the method further includes modifying the inserted data block to include the insertion attribute.
In some examples, the method further includes transmitting the insertion attribute.
In other examples, the insertion attribute includes information associated with the priority order, a priority of the decoding, a timestamp, and/or information associated with the portion of the data.
In some examples, the method further includes encoding the multimedia material block.
In other examples, the method further includes encoding the inserted data block.
In some examples, the encoding is a forward error correction encoding.
In other examples, the priority order is an inverse time recording order, a time recording order, a data portion size order, and/or a data type order.
In some examples, the priority order is a predetermined priority order.
In another aspect, an apparatus includes a transceiver and an interposer. The transceiver is configured to receive a multimedia data block and to transmit an inserted data block, the multimedia data block including a plurality of data portions. The inserter is configured to insert the plurality of data portions to form the inserted data block based on a prioritization based on a multimedia material type of the data portion.
In some examples, the device further includes the inserter being additionally configured to generate an attribute based on information associated with the inserted data block.
In other examples, the device further includes the inserter configured to modify the inserted data block to include the insertion attribute.
In some examples, the device further includes the transceiver configured to communicate the insertion attribute.
In other examples, the device further includes an encoder configured to encode the multimedia data block.
In some examples, the device further includes an encoder configured to encode the inserted data block.
In other examples, the device further includes the inserter, which is additionally configured to generate the prioritization based on a multimedia type associated with the data block.
In another aspect, a computer readable storage medium encoded with instructions for executing a multimedia data block when executed by a computer, the multimedia data block including a plurality of data portions; The plurality of data portions form an inserted data block based on a priority order, the priority order being based on a multimedia material type of the data portion; and transmitting the inserted data block.
In another aspect, a method includes receiving one or more data portions of an inserted multimedia data block; based on time information and/or one or more data associated with the one or more data portions Part of the data type information determines a first data portion of the one or more data portions; and is determined in a time interval of the first data portion based on playback time information of an earliest data portion of a subsequent data block One-time project.
In some examples, the method further includes configuring the one or more data portions based on time information associated with the one or more data portions and/or data type information of the one or more data portions.
In other examples, the method further includes playing one or more portions of the data of the configurations.
In some examples, the method further includes playing the first portion of data based on the schedule item in the time period.
In other examples, the method further includes playing the first portion of data, the method further comprising displaying the first portion of data and/or restoring the portion of the first portion of data.
In some examples, the inserted multimedia data block includes an audio data portion, a video data portion, an image data portion, a text data portion, and/or an animation data portion.
In other examples, the method further includes determining the playback time information based on the segment received by one of the inserted multimedia data blocks.
In other examples, the method further includes determining, based on the playback time information and/or freeze time information of the earliest data portion of the subsequent data block, the time history item in the time period of the first data portion.
In some examples, the method further includes determining the freeze time information based on a difference between time information associated with the next multimedia material block and time information associated with the one or more data portions.
In other examples, the method further includes determining a first portion of the one or more data portions based on an insertion attribute.
In some examples, the insertion attribute includes information associated with a sequence of the one or more data portions of the inserted multimedia data block.
In some examples, the data type information of the one or more data portions includes a multimedia type of the one or more data portions.
In another aspect, a device includes a transceiver and a media processor. The transceiver is configured to receive one or more data portions of an inserted multimedia data block. The media processor is configured to determine a first of the one or more data portions based on time information associated with the one or more data portions and/or data type information of the one or more data portions The data portion determines a time-history item in the time course of the first data portion based on the playback time information of the earliest data portion of the subsequent data block.
In some examples, the device further includes the media processor configured to be based on time information associated with the one or more data portions and/or data type information of the one or more data portions Configure one or more of the data sections.
In other examples, the device further includes a display configured to play one or more portions of the data of the configurations.
In some examples, the device further includes a display configured to play the first portion of data based on the schedule item in the time period.
In other examples, the device further includes the media processor, which is additionally configured to determine the playback time information based on a segment received by one of the inserted multimedia data blocks.
In some examples, the device further includes the media processor, the other configured to determine the time of the first data portion based on the playback time information and/or freeze time information of the earliest data portion of the subsequent data block The time project in the process.
In some examples, the device further includes the media processor configured to be based on a difference between time information associated with the next multimedia material block and time information associated with the one or more data portions To decide the freeze time information.
In other examples, the device further includes the media processor, which is additionally configured to determine a first portion of data of the one or more data portions based on an insertion attribute.
In another aspect, a computer readable storage medium encoded with instructions for performing one or more data portions of an inserted multimedia data block when executed by a computer; based on being associated with the The time information of one or more data portions and/or the data type information of the one or more data portions determines the first data portion of the one or more data portions; and is based on an earliest one of the subsequent data blocks The playback time information of the data portion determines a one-time project in the first time portion of the first data portion.
The insertion technique of one of the multimedia data blocks described herein can provide one or more of the following benefits. One benefit is to reduce transmission errors and/or packet loss while providing a better viewing experience for the user. Another benefit is the sequentialization of the inserted data so that multimedia reception is enabled when the remaining multimedia is received, providing better viewing experience for the user, as well as fast channel switching. An additional benefit is the longer viewing duration of the inserted data, providing better viewing experience for the user, as well as fast channel switching.
The advantages and features of these and other specific embodiments, as well as the organization and manner of the operation, will be more apparent from the following detailed description of the accompanying drawings.
In the following description, for purposes of explanation and description However, it will be understood by those skilled in the art that the specific embodiments may be practiced in other embodiments.
This case contains Annex A. Annex A includes an illustration of an exemplary embodiment of the present application, which is incorporated herein by reference.
Figure 1 is an example communication system 100 for data transfer. The communication system 100 includes at least one communication device 110, a communication network 120, and at least one multimedia server, such as servers A130a, B130b to Z130z, commonly referred to as 130. The communication device 110 includes a wireless telephone 110a, a personal digital assistant (PDA) 110b, a computer 110c, a television 110z, and/or the like. Those skilled in the art will appreciate that a communication device 110 can be any communication device.
During a multimedia streaming session, the communication device 110 receives the multimedia source block from the one or more multimedia servers 130 via the communication network 120. A multimedia source block contains a plurality of data units or data packets. A data package containing media content can carry information representative of the order in which the content is presented (eg, a timestamp) in the data packets. The communication device can decode the media and display it in a time-recorded order defined by the presentation order information.
According to an exemplary embodiment, the data portions of the multimedia source block (e.g., data elements or data packets) are inserted, ordered, or configured in a different order than their corresponding presentation order. For example, the portion of the data in the multimedia source block can be inserted or configured by reducing the turn-in time on the receiving side. In an exemplary embodiment, different portions of data, such as in a multimedia source block, may be assigned different priorities depending on where those data portions are inserted. In another exemplary embodiment, a portion of the data used to decode other portions of the data, such as in the same source block, is placed at the end of the multimedia source block. This insertion/configuration allows a receiving device to quickly transfer in and begin displaying media content. For example, the portion of the data associated with the audio material may also be placed at the end of the multimedia source block to allow the communication device to, for example, join a multimedia streaming session to begin playing the audio content without undesired delays.
In an exemplary embodiment, communication device 110 receives embedded data blocks, such as multimedia source blocks, from multimedia server 130 via communication network 120 and processes the inserted data blocks for viewing and/or storage. When transferring to the multimedia streaming session, the communication device 110 can begin receiving the data portion at a random location within a data source block. The receiving communication device 110 inversely inserts/reconfigures the inserted data portions based at least in part on time information (e.g., time stamps) associated with the one or more received data portions. The de-insertion or re-arrangement is ordered in the presentation/playback order of the data portions. For example, the de-insertion is the reverse of the insert procedure. The communication device 110 can, for example, decode the inversely inserted data portions. According to an exemplary embodiment, the communication device 110 determines a time-history item of the first data portion of the first data portion, such as the time at which playback is to begin, among the inversely inserted data portions. The time-course item is determined, for example, based at least in part on playback time information and/or a minimum buffer time of the first and last data portions.
When the unicast and multicast material delivery is transmitted while the multimedia material is being transmitted, the communication device 110 communicates with the multimedia server 130 via the communication network 120 to request the multimedia material. The multimedia server 130 receives the requested multimedia material from the communication device 110. When using unicast material delivery, the request can identify the multimedia material to be transmitted, including the starting location within the content item, such as a collection of television albums. When multicast material delivery is used, the multimedia server 130 is transmitting one or more multimedia streams, and the request substantially identifies that the communication device 110 wants to receive those in the transmitted multimedia stream. The multimedia server 130 receives a multimedia material block associated with the requested multimedia material. When unicast material delivery is used, the multimedia material block corresponds to the requested multimedia material, which includes a starting location within the content item. When multicast data delivery is used, the multimedia data block is the next data block to be streamed for each multimedia material.
When the broadcast material delivery is used to transmit the multimedia material, the communication device 130 can begin receiving the multimedia material at any location where it is transmitted. The multimedia server 130 receives a multimedia data block that is the next data block to be transmitted.
The multimedia data block includes a plurality of data sections. The multimedia server 130 inserts the plurality of data portions to form an inserted data block. According to an exemplary embodiment, the plurality of data portions are inserted based on a prioritized order, such as an opposite decoding order, a decoding dependency, a reverse order of an extensibility layer, and/or the like. The insertion preferably results in improved viewing experience for the user to provide the first received data block in the communication device 110, such as a shorter multimedia start time, a shorter blank screen time, and more. The audio of time and so on. The multimedia server 130 transmits the inserted data block to the respective communication device 110.
In some examples, the multimedia server 130 receives a stream of multimedia data comprising a plurality of multimedia material blocks from a multimedia storage server (not shown) based on a request from the multimedia server 130. In other examples, the multimedia server 130 receives the local storage (eg, a hard drive, a CD player, an optical jukebox, etc.) and/or a remote storage (eg, a storage area network, network attached storage, etc.) Multimedia streaming. In some examples, multimedia server 130 receives the stream of multimedia data from a broadcast server (not shown) that broadcasts live multimedia programs (e.g., television programs, movies, etc.).
For example, a user associated with computer 110c transfers to a broadcast/multicast session of a television program, or requests to watch a television program, such as "Dogs of Big Wars." The computer 110c may locally join a broadcast session, transmit the request to join the multicast session to a server via the communication network 120, or transmit the unicast request to the multimedia server B 130b via the communication network 120. If it is a multicast delivery, the server sets the streaming pass to the computer 110c; then the streaming server B 130b begins streaming the television program to the computer 110c. If it is a unicast delivery, the multimedia server B 130b processes the requests and begins streaming the television program to the computer 110c. Streaming television programs by multimedia server B 130b includes several steps. The multimedia server B 130b, for example, retrieves or receives a multimedia data block of the television program from the local storage. The multimedia data block includes a plurality of data portions, such as audio, video, video, control data, and the like. The multimedia server B 130b inserts the plurality of data portions to form an inserted data block. In this example, the portions of the data associated with the audio are grouped together at the end of the data block, and the portions of the data associated with the video are grouped together at the beginning of the data block. The order of the data portions is based on the priority order of the multimedia type, that is, the video is first and the audio is second. The multimedia server B 130b transmits the inserted data block to a multicast group, which is associated with a multicast or broadcast session, or to the computer 110c, for example, for a unicast session.
In another example, a user associated with computer 110c requests to watch a television show ("Dogs of Big Wars" in this example). The computer 110c transmits the request to the multimedia server B 130b via the communication network 120. The multimedia server B 130b processes the requests and sets the computer 110c in a multicast group, i.e., the communication devices that have requested the television program to be viewed. The multimedia server B 130b begins transmitting the television program to the computer 110c by multicasting to the multicast group. In other words, the multimedia server B 130b initiates the transfer to the computer 110c, which is the same data block as the other communication devices in the multicast group. Streaming television programs by multimedia server B 130b includes several steps. The multimedia server B 130b receives a multimedia data block of the television program. The multimedia data block includes a plurality of data portions (eg, audio, video, video, control data, etc.). The multimedia server B 130b inserts the plurality of data portions to form an inserted data block. In this example, the portions of the data associated with the audio are grouped together at the end of the block, and the portions of the data associated with the video are grouped together at the beginning of the block. The order of the data portions is based on the priority order of the multimedia type (ie, video first, and audio second). The multimedia server B 130b transmits the inserted data block to the communication devices in the multicast group, that is, multicasts the television program to all communication devices.
In another example, a user associated with computer 110c transfers (eg, changes channels) to watch a television show ("Dogs of Big Wars" in this example). The multimedia server B 130b is transmitting the television program to the channel, ie broadcasting the television program, and the computer is transferred to the channel. The broadcast of the television program by the multimedia server B 130b includes several steps. The multimedia server B 130b receives a multimedia data block of the television program. The multimedia data block includes a plurality of data portions (eg, audio, video, video, control data, etc.). The multimedia server B 130b inserts the plurality of data portions to form an inserted data block. In this example, the portions of the data associated with the audio are grouped together at the end of the block, and the portions of the data associated with the video are grouped together at the beginning of the block. The order of the data portions is based on the priority order of the multimedia type (ie, video first, and audio second). The multimedia server B 130b transmits the inserted data block to the channel, such as a television broadcast channel, an internet broadcast channel, and the like.
In yet another example, computer 110c receives the one or more data portions of the inserted data block from multimedia server B 130b. The communication device 110 determines a first data portion of one or more of the one or more data portions. The first portion is determined based on the first complete multimedia segment (e.g., audio, video, video, etc.) received by computer 110c. In this example, computer 110c receives only half of the portions of the video material, but has received all of the audio data portion. Because the first full multimedia segment is the audio segment, computer 110c determines that the first portion is the beginning of the portion of the audio material. The computer 110c determines a time-history item in the time course of the first data portion (ie, the first audio data portion). The time history item is determined based on the playback time information and/or freeze time information (in this example, the freeze time information includes a one second delay between the audio data portion and the next data block) ). In this example, the time-course item begins playing the portions of the audio data after one second such that the end of the end of the audio data portion is the same as the beginning of the playback of the next data block.
Although there are four communication devices 110 and a communication network 120 as shown in FIG. 1, the system 100 can communicate with a plurality of communication devices via a plurality of interconnected networks (eg, an Internet, an intranet, or an extranet).
Figure 2A shows a block diagram of an example communication device 210 in communication system 200a. The communication device 210 includes a transceiver 211, a complex decoder 212, a media processor 213, a display 214, and a storage device 215. The transceiver 211 receives and/or transmits data (eg, data portions, data blocks, data streams, packets, etc.). The complex decoder 212 decodes the received data (e.g., decodes a block code, decodes a convolutional code, decodes the media data, decrypts the stream, etc.). The media processor 213 determines a first portion of one or more of the data portions and/or a time-history item that determines the presentation of the first data portion. Display 214 displays the data (eg, video, video, etc.). The storage device 215 (eg, a memory, a hard drive, etc.) stores the material.
Although the decoder 212 shown in FIG. 2A and the media processor 213 are different components, the communication device 210 can use the decoder 212 to decode the received data and determine the first portion of the one or more data portions. Decide on a time-course project in one of the first data sections.
Figure 2B shows a block diagram of an example multimedia server 230 in communication system 200b. The multimedia server 230 includes a transceiver 231, a source encoder 232, a multiplexer 233, a Forward Error Correction (FEC) encoder 234, an interpolator 235, and a storage device 236. The transceiver 231 receives and/or transmits data (eg, data portions, data blocks, data streams, packets, etc.). Source encoder 232 encodes the received data. The multiplexer 233 is multiplexed (for example, in combination with the received data and other received data, combined with various types of data for transmission, etc.). The FEC encoder 234 encodes the data. The inserter 235 inserts (e.g., configures the data portions in a non-contiguous configuration, changes the order of the portions of the data, and reconfigures the order of the portions of the data). The storage device 236 (eg, memory, hard drive, etc.) stores the data.
In some examples, media processor 213 configures the one or more data portions based on time information associated with the one or more data portions and/or data type information for the one or more data portions. For example, media processor 213 configures the portions of data such that the portions of data are no longer inserted, but in a prioritized order, such as for prioritization, such that the user can view quickly. For many encoded media streams, the order of precedence is the same as a time-recorded presentation order. In other examples, display 214 plays the portion of the data of the configurations in accordance with the time-recorded presentation sequence.
FIG. 3 is a detailed block diagram of another example system 300 including a multimedia server 330. System 300 includes multimedia storage servers A 325a, B 325b through Z 325z (collectively 325), and a multimedia server 330. The multimedia server 330 includes source encoders A 331a, B 331b through Z 331z (collectively 331), a multiplexer 333, a forward error correction (FEC) encoder 335, and an interpolator 336.
Each source encoder 331 receives data from a multimedia storage server 325 (eg, a SAN, NAS, a third party website, a third party storage server, etc.). The source encoder 331 encodes (e.g., error correction code, protocol conversion, etc.) the data to form the data streams A 332a, B 332b through Z 332z (collectively 332), respectively. The multiplexer 333 multiplexes the stream 332 to form a source block 334. The source block 334 includes a portion of the multiplexed data stream (eg, the source block 334 includes 1024 symbols of the multiplexed data stream, and the source block 334 includes 512 octets of multiplex. Data stream, etc.). Source block 334 includes the multiplexed data streams. The FEC encoder 335 encodes the source block 334 to form the patch material 337b, and the inserter inserts the encoded source block to form an inserted source block 337a.
Although the FEC encoder 335 and the interpolator 336 shown in FIG. 3 are separate components, the interpolator 336 can be incorporated into the FEC encoder 335. In other words, the FEC encoder 335 encodes and inserts the source block.
FIG. 4 is a detailed block diagram of another example system 400 including a multimedia server 430. System 400 includes multimedia storage servers A 425a, B 425b through Z 425z, and a multimedia server 430. The multimedia server 430 includes source encoders A 431a, B 431b to Z 431z, a multiplexer 433, a forward error correction (FEC) encoder 435, and an interpolator 436.
Each source encoder 431 receives data from a multimedia storage server 425 (eg, a SAN, NAS, a third party website, a third party storage server, etc.). The source encoder 431 encodes (e.g., error correction code, protocol conversion, etc.) the data to form the data streams A 432a, B 432b through Z 432z (collectively 432), respectively. The multiplexer 433 multiplexes the stream 432 and the interpolator 436 inserts the multiplexed streams to form an inserted source block 434. The inserted source block 434 includes a portion of the multiplexed data stream (e.g., source block 434 includes 1024 symbols of the multiplexed data stream, and source block 434 includes 512 octets. Such as multiplexed data stream, etc.). The FEC encoder 435 encodes the inserted source block 434 to form an encoded source block 437a and patch data 437b. The encoded source block 437a is identical in content to the inserted source block 434.
In some examples, the patch material 337b is utilized by the communication device 210 of FIG. 2A to correct erroneous and/or lost material in the encoded source block 437a. The communication device 210 can decode the encoded source block 437a using forward error correction decoding.
In another embodiment, the interposer 436 is located before the multiplexer 433.
Figure 5A shows a block diagram 500a of an example multimedia source block 534a. Block diagram 500a includes source block 534a, interpolator 536a, and an inserted source block 537a. The source block 534a includes an independent access unit (IDR), an audio material ("A"), a reference access unit ("P"), and a non-reference access unit ("p"). The inserter 536a is inserted into the source block 534a to form the inserted source block 537a. As shown in the inserted source block 537a, the interposer 536a group IDR, A, P, and p data portions are together. In this example, the IDR is at the end of the inserted data block because the IDR provides an image segment of the multimedia content that can be preferably played (eg, displayed, etc.) to the user during the transition time. . In this example, A is immediately adjacent to the inserted data block because the audio material provides audio of the multimedia content, which may preferably be played to the user during the transition time of the channel change (eg, Through the speakers, through the headphones, etc.).
In some examples, the insertion of source block 534a does not increase the time to play the multimedia because the reverse insertion delay is covered by the lowest buffer time, as described below. One advantage is that multiple multimedia material types can prioritize transmissions in each data block so that the communication device 210 of Figure 2A can be played more quickly and provide better viewing experience for the user (ie, Less waiting time when the channel changes).
In other examples, the interpolator 536a inserts the source block 534a based on the media type of the source block 534a (ie, the interposer 536a is a known medium). For example, interposer 536a distinguishes between independent, reference, and non-reference media units in source block 534a. In another example, in a mobile TV service, the interposer 536a inserts the audio material A at the end of the source block. The insertion based on the media type preferably results in the success and early playback of the audio material A, which can significantly enhance the user's experience. In another example, the inserter 536a inserts the video material in such a manner that an independent image IDR is decoded and displayed by placing it at the end of the inserted source block 537a. Placing the independent image IDR at the end of the inserted source block 537a preferably results in the display of an initial image and avoids displaying a full black screen. In some examples, the interposer 536a organizes the reference media units P in an inverse encoding order that increases the amount of decodable material that the communication device 210 can use.
The benefit of the non-inserted data stream is that the inserted data stream provides a longer media duration for the communication device 210. Another benefit is that the additional media duration reduces overall channel switching delay and allows a communication device to enhance the user experience (e.g., display of a still image, play the audio, etc.) by providing basic playback of the multimedia.
Figure 5B shows a block diagram 500b of an example multimedia source block 534b. Block diagram 500b includes source block 534b, an interpolator 536b, and an inserted source block 537b. The source block 534b includes a Scalable Video Coding (SVC) streaming media unit having three layers: a base layer (BL) and a first enhancement layer (EL1). And a second enhancement layer (Se2). Source block 534b additionally includes audio material ("A"). Inserter 536b is inserted into source block 534b to form an inserted source block 537b. As shown in the inserted source block 537b, the interpolator 536b configures the EL2 data unit to be transmitted first, followed by the EL1 data unit, and finally the BL and A data units. This prioritization allows the communication device 210 of Figure 2A to partially receive the inserted source block 537b to properly replay the A and BL units in an early playback procedure.
Figures 6a and 6b show a timeline 620 of an exemplary scheduling program 600a and 600b for receiving a communication device. The scheduling programs 600a and 600b collectively include source blocks i610a, i+1 610b, and i+2 610c, a roll-in time 622, and a minimum buffer time 624. The scheduler 600a includes an early playback start 626a and a freeze period 628. Scheduler 600b includes an early playback start 626b.
As shown in Figures 6a and 6b, timeline 620 is the same. However, Figure 6b shows the decision and correction of the freeze period 628 by the communication device 210 of Figure 2A. The freeze period 628 occurs because the first source block i 610a is not completely received by the communication device 210. The first segment 614a of the source block i 610a is not received by the communication device 210 because the communication device 210 does not transfer to 622 until after the first segment 614a is transmitted. The second section 616a of the source block i610a is received by the communication device 210. Therefore, the communication device 210 receives only the second segment 616a of the source block i 610a. The communication device 210 determines that the second segment 616a can be played at the early playback start time 626a. However, if the second segment 616a begins at the early playback start time 626a, the freeze period 628 will occur (i.e., the time region between the last data portion of the second segment 616a and the beginning of the source block i+1 610b). segment).
In order to determine the freeze period 628 and how to modify the freeze period 628 (ie, no freeze period, lower freeze period, etc.), the communication device 210 can make several decisions and/or calculations, as described below. The communication device 210 can determine the amount of data it receives from the current source block (the second segment 616a in this example). For example, communication device 210 determines the difference between the highest and lowest received time stamps of the data portions (eg, using an extended RTP timestamp representation) and converts the difference to seconds and divides by a media clock rate. Decide on the amount of information received.
In some examples, communication device 210 determines the portion of the first data to be played back. For example, the first portion is the earliest audio media portion (eg, using audio as a reference for the presentation, etc.), the earliest media portion (eg, using presentation time, using timestamps, etc.), or the latest media unit. In other examples, the communication device 210 is based on the reception time of the first media portion of the source block and based on the measurement of the end of the first media portion in the next source block (ie, source block i+1 610b) The minimum buffer time to determine the playback time of the earliest media portion of the next source block.
In some examples, the communication device 210 schedules based on the playback time information (eg, the duration of the first data portion, the duration of the first data portion and the subsequent data portion, etc.) and/or the calculated freeze period The first media portion from the current source block 610a. For example, the playback of the first media unit of the communication device 210 scheduling source block i+1 610b is the minimum buffer time after its reception time, and the second region of the source block i 610a at the early playback start 626b. Segment 616a. In other examples, the communication device 210 determines the early playback start 626b because a late start may cause a delay in the playback of the source block i+1 610b, and an early start may cause the buffer to be under-subscribed and cannot be played back to use. By.
In other examples, communication device 210 determines the freeze time information (eg, the duration of the freeze point, the start time of the freeze point, etc.). The freeze point duration is calculated as the time difference between the lowest timestamp of the successor source block and the highest timestamp of the available media portions. In some examples, the latest timestamp of the current block is sent via an insertion attribute (eg, in the inserted data block, via a signaling channel, etc.), and the latest timestamp is used Determine the lowest timestamp of the successor source block. One benefit is that the data block can optimally schedule early playback from the second segment 616a of source block i 610a.
In some examples, the minimum buffer time 624 represents the time required to ensure uninterrupted playback. The communication device 210 determines the minimum buffer time (a portion of the playback time information). The communication device 210 determines the minimum buffer time based on the time between receipt of the first packet of the current source block (ie, the first data portion) and the start of the media decoding. In other examples, the starting point for media decoding is the earliest among the source packets received in priority order.
In other examples, the minimum buffer time 624 is utilized by a service provider (i.e., the provider hosting the service provided by the multimedia server 230) to signal the minimum time to wait until the received FEC decoding is to be triggered before a source After the first packet of the block. The service provider can ensure that, for example, no source block (eg, source block i+1 610b) and its patching data will be greater than the "lowest buffer time" value of the message.
In some examples, the error correction protection provided by the FEC encoding/decoding service can cause a significant delay that is necessary for receiving and correctly FEC decoding a complete source block. In other examples, the minimum buffer time 624 is determined based on the estimated transmission time of the largest source block (including FEC data) to be used. In some examples, communication device 210 receives an instruction to wait for at least the lowest buffer time 624 of the transmission before the FEC decoding begins.
In some examples, communication device 210 begins to play media for the same service at different times. For example, communication device 210 plays a television advertisement while communication device 210 is waiting to receive the television program. The communication device 210 can determine which other media to play based on the amount of media material available for each service and/or at the beginning and end of the playback curve (e.g., as shown in Figure 7a).
In other examples, the lowest source block size is 1024 source symbols. For example, a source signal of size 512 bytes corresponds to the overall source block size of 4,194,304 bits. In another example, the bit rate of an MBMS service is 256 kbps, which would result in a source block covering approximately 16 seconds of data. In some examples, each source block is constructed from a target source block that is one of 80,000 bytes in size.
In some examples, communication device 210 transitions to an MBMS service at a random point in time (e.g., roll-in time 622), and an average transfer time is centered in a source block. The communication device 210 initiates a timer of the duration of the minimum buffer time 624 to begin decoding. In other examples, the received partial block data (e.g., second segment 616a) cannot be decoded (e.g., incomplete data, lost material, etc.), which can result in unsatisfactory video quality or additional delay. The communication device 210 can, for example, decide how to smooth out the unsatisfactory video quality and/or fill up the audio/video for the delay.
In other examples, communication device 210 reorders the unordered packets, and the insertion is transparent to other layers of communication device 210.
Figure 7A shows a graph 700a of the presentation/display timestamp of the data portion. The graph 700a includes a timestamp of the source packet in seconds along the x-axis 710a and a source packet transmission sequence along the y-axis 720a. As shown, the low priority data is transmitted at the beginning of source block 732a, the high priority data is transmitted in the reverse presentation order at the end of source block 734a, and the very high priority data is at the source block 736a. The end is transmitted. For example, the source block 537a inserted in FIG. 5A shows the order of the data in the graphic 700a, the non-reference image "p", and the reference image "P x ", the audio data "A" and the IDR map in the opposite order. Like "IDR".
Figure 7B shows a graph 700b of the presentation/display timestamp of a non-inserted transmission of a data block. The graph 700b includes a presentation timestamp of the source packet in seconds along the x-axis 710b and a source packet transmission sequence along the y-axis 720b. As shown, the data (e.g., as shown in source block 534a of Figure 5A) is continuously transmitted in a transport stream 730b.
In some examples, Figures 7A and 7B illustrate the benefits of adding media data insertion under the media data that is usable and correctly decodable at the time of transfer. In some examples, the interpolator 235 of FIG. 2B classifies a media packet of an H.264 media stream (encoded at 300 kbps) into a packet carrying an IDR image segment, a reference P image segment, and a non-reference p image.
For example, the interpolator 235 configures the "IDR" image segments at the end of the source block with a rise time priority, as shown by the inserted source block 537. The reference "P" image data portions are transmitted at the end of the inserted source block 537 (before the IDR image packet) in descending presentation time order. Finally, the non-reference "p" image packets are transmitted at the beginning of the inserted source block 537 with a rising presentation time.
FIG. 8 shows an example communication device 810 for use in the example system 800. System 800 includes a communication device 810, a network 820, and a multimedia server 830. The communication device 810 communicates with the multimedia server 830 via the network 820. The communication device 810 includes an antenna 851, a housing 852, a speaker 853, a display 854, a keyboard 855, a microphone 856, a memory 857, and a battery 858. Display 854 displays an interactive display 864 for selecting multimedia content.
Antenna 851 is used to receive and/or transmit data signals and may be constructed from any known antenna material. The housing 852 is a housing for housing the components of the communication device 810 (e.g., the components shown in the communication device 210 of Figure 2A, the components shown in the communication device 810, etc.). The outer casing can be constructed from any known outer casing material (e.g., plastic, metal, etc.). The speaker 853 is used to reproduce the data portion (for example, an audio data portion, etc.). The communication device 810 can include, for example, a speaker output (not shown) for use by an external speaker (e.g., earphone, third party speaker, wireless connection speaker, etc.) for reproducing the data portion.
Display 854 is used to play the portion of the data. Display 854 can display such portions of data (eg, display such images, etc.) and/or reproduce portions of such data (eg, tactile touches of such animations, etc.). Display 854 displays an interactive display 864 for selecting one of the multimedia content. A user associated with the communication device 810 can select the multimedia content to play. The communication device 810 transmits a request for the multimedia content based on the multimedia content selected to be played.
Keyboard 855 is used to enter selections and/or other input information (eg, name, phone number, etc.). The microphone 856 is used to input audio data (such as voice calls, commands, audio recordings, etc.). The communication device 810 can include, for example, a microphone output (not shown) for inputting audio by an external microphone (e.g., earphone, third party microphone, wireless connection microphone, etc.). The storage 857 is used for storing data (for example, storing multimedia data portions, obtaining multimedia data portions, storing telephone numbers, etc.). The storage 857 can be any type of memory storage, including a removable memory storage and/or a permanent memory storage. Battery 858 is used for power to communication device 810. Battery 858 can be any type of electrical device (eg, a rechargeable battery, a disposable battery, etc.).
Figures 9A and 9B are examples of data block descriptions 900a and 900b including insertion descriptors 920a, 920b and 925b. The data block description 900a includes general descriptors 910a and 930a of the data block and an insertion descriptor 920a. The data block description 900b includes general descriptors 910b, 930b, and 935b and insertion descriptors 920b and 925b of the data block.
In some examples, Figures 9A and 9B illustrate the transmission of an Insert using a Service Description Protocol (SDP) attribute. The SDP attribute can be included in the SDP description of the FEC stream. The SDP attribute can be a media level attribute and can be applied to all streams protected by FED streams (e.g., stream 332, stream 432).
Figure 9A shows the inserter property 920a, "X-3gpp-Interleaved". As shown, the augmented backup-naur form (ABNF) syntax for this attribute can be defined as follows: Interleaving=”a=X-3gpp-Interleaved:”SP[“true”/“false”] CRLF
In other examples, the signaling may include more detailed information regarding the location and reordering of the media portion of each data stream. For example, the signaling includes media level attributes 920b and 925b in the corresponding media definitions of the SDP file. Figure 9b shows the media level attributes of video stream 920b and audio stream 925b. In this example, the video stream is designated as priority 1, and the audio stream is designated as priority 0. Based on this preference, the interpolator 235 inserts the audio material at the end of the source block prior to the video material.
In some examples, the insertion attribute includes an inter-stream configuration attribute. For example, the inter-stream configuration attributes include mixed, original, and vice versa. The blending may represent the transmission sequence including both the original and reversed sequence of media portions. The original can represent that the transmission order has not changed. This opposite can mean that the transmission order is reversed.
In other examples, the ABNF syntax for inter-stream configuration properties can be as follows:
Interleaving-Info=”a=X-3gpp-Interleaving:”Priority SP[“original”/“reverse”/“mixed”]CRLF Priority=1*DIGIT
In some examples, the inter-stream configuration attribute is placed in the SDP archive of the FEC stream. In this example, the inter-stream configuration attribute references the flow ID to identify the referenced stream.
In other examples, the time information associated with the portion of the data is transmitted in a subset of the RTP packets of the stream in a pattern that is expanded by the RTP header. For example, the identification of the RTP header extension representing a 3gpp insertion header extension follows the 32-bit representation of the highest timestamp of the current source block. In another example, the source block number is passed by the source FEC payload, which is appended to each packet (also referred to as a data portion).
In some examples, communication device 210 determines that the primary audio stream of the MBMS service is in its original order based on the insertion attribute and/or analysis of the data block (eg, at the beginning of the data block, at the data block) Pass at the end, etc.). In this example, the last received audio media portion carries the highest timestamp of the source block. The highest audio timestamp value can be used to determine the corresponding video timestamp.
In other examples, the insertion of the data block is backward compatible. In other words, for example, today's communication devices are instructed to restore the original order of the media packets based on the Real-time Transport Protocol (RTP) sequence number value. In some examples, the insertion does not change the RTP sequence number of the inserted stream. Based on the lack of a change in the RTP sequence number, the multimedia server 230 does not utilize, for example, a reordering mechanism (e.g., decoding sequence numbering, etc.).
In some examples, communication device 210 utilizes the insertion attributes to reduce the roll-in time. The communication device 210 can adjust the playback curve of the media unit of the source block received by the first portion based on the insertion attributes. The processing of the subsequent source block may be equivalent to the situation when the communication device 210 is transferred at the beginning of the source block. In other words, the FEC corrected minimum buffer time for a source block can be applied from the first received media unit of the next FEC block.
Figure 10A is a flow chart 1000a of an exemplary insertion procedure performed by the multimedia server 230 of Figure 2B. The transceiver 231 receives (1010) a multimedia data block. The multimedia data block includes a plurality of data portions (also referred to as media units). The inserter 235 determines (1030) a timestamp of a last data portion based on information associated with the last data portion (e.g., SDP stream information, a timestamp signal, etc.). The inserter 235 inserts (1020) the portions of the material of the multimedia material to form the inserted data block. The inserter 235 modifies (1040) the inserted data block to include the timestamp of the decision. The transceiver 231 transmits (1050) the inserted data block.
In other embodiments, the communication device 210 of FIG. 2A receives the inserted data block and the determined time stamp via the transceiver 211. The media processor 213 can reorganize the inserted data block into the data block and identify the timestamp. The media processor 213 can utilize the timestamp to determine which packet to start playing, as described herein.
Figure 10B is a flow chart 1000b of an exemplary insertion procedure performed by the multimedia server 230 of Figure 2B. The transceiver 231 receives (1080) a multimedia data block. The multimedia data block includes a plurality of data portions (also referred to as media units). The inserter 235 inserts (1082) the portions of the material of the multimedia material to form the inserted data block. Transceiver 231 transmits (1084) the inserted data block.
In other embodiments, the communication device 210 of FIG. 2A receives the inserted data block via the transceiver 211. The media processor 213 can reorganize the inserted data block into the data block. The media processor 213 can determine the media type of the data block, such as audio, video, and/or the like. The media processor 213 can determine the first and last packets in the data block and identify the timestamps of the first and last packets. The media processor 213 can utilize the difference between the timestamps of the first and last packets to determine which packet to start playing, as described herein.
In other embodiments, the communication device 210 of FIG. 2A receives the inserted data block via the transceiver 211. The media processor 213 can reorganize the inserted data block into the data block. The media processor 213 can determine the media type of the data block, such as audio, video, and/or the like. The media processor 213 identifies the timestamp of one or more audio packets. The media processor 213 can utilize the timestamps of the one or more audio packets to determine which packet to start playing, as described herein.
Figure 11 is a flow diagram 1100 of an example scheduling procedure performed by communication device 210 of Figure 2A. The transceiver 211 receives (1110) one or more data portions of an inserted multimedia data block. The media processor 213 reversely inserts (1120) the data portions. The media processor 213 determines (1130) the data type information of the data portions. The media processor 213 determines (1140) time information associated with the data portions.
The media processor 213 determines (1170) freeze time information and determines (1180) a time-history item of a data portion based on the freeze time information. Display 214 plays (1185) the portion of the data based on the schedule item.
In some examples, the insertion may provide better results when applied to FEC protected scalable video coding (SVC) streaming. In a multicast environment, SVC will have more benefits when transmitting in a multiple streaming mode. SVC has more advantages because the base layer is transmitted during the RTP session independent of the promotion layers, and one or more enhancement layers can be transmitted during an RTP session, and temporal scalability can be combined with each A spatial layer is used together to produce one or more time dependent sublayers.
In other examples, the SVC and RTP payload formats provide the syntax elements (eg, priority_id, dependency_id, quality_id, temporal_id, etc.) of the interpolator 235 of FIG. 2B in the NAL unit header for transmission of the media units. Sequence reconfiguration. For example, a higher priority media unit may be transmitted towards the end of the source block, and/or the base layer may be transmitted in reverse time recording order. In some examples, information about layer priority may be retrieved from the SDP file, so resolution of the NAL unit header and extension is not required.
In some examples, communication device 210 determines the insertion of the data block and forwards the media packet to media decoder 212 before the "minimum buffer time" protection period of the call. In this example, the channel transfer time is preferably reduced by using the properly reconstructed media material to begin playback as quickly as possible.
In other examples, the communication device 210 that is not capable of determining the insertion of the data block preferably reduces the transfer time by utilizing the media packets at the end of the source block, which causes the media material to be The correct re-construction can be done.
The various embodiments described herein can be implemented as software, hardware, application logic, or a combination of software, hardware, and application logic. The software, application logic and/or hardware may be present, for example, on a chipset, a mobile device, a desktop computer, a laptop, or a server. Software and web page implementations of various embodiments can be implemented using standard stylization techniques with regular logic and other logic to perform multiple database search steps or procedures, associated steps or procedures, comparison steps or procedures, and decision making Step or program. Various embodiments may also be implemented, in whole or in part, in a network element or module. It must be noted that the terms "component" and "module" as used herein and in the scope of the following claims are intended to cover the use of one or more lines of software code and/or hardware. And/or the implementation of a device that receives manual input.
The above systems and methods can be implemented as digital electronic circuits, computer hardware, firmware and/or software. The implementation can be implemented as a computer program product (for example, a computer program embodied on an information carrier). The implementation can be performed, for example, in a machine readable storage, executed by or controlled by the data processing device. The implementation can be, for example, a programmable processor, a computer, and/or multiple computers.
A computer program can be written into any type of stylized language, including compiled and/or interpreted languages, and the computer program can be deployed in any form, including as a separate operating program or as a sub-example, component and / or other units suitable for use in a computing environment. A computer program can be deployed to be executed on a computer or on multiple computers at a location.
Method steps may be performed by one or more programmable processors executing a computer program to perform the functions of the present invention by operating input data and generating output. The method steps can also be performed by a device that can be implemented as a special purpose logic circuit. The circuit can be, for example, a Field Programmable Gate Array (FPGA) and/or an Application Specific Integrated Circuit (ASIC). Modules, sub-examples, and software agents may represent portions of computer programs, processors, special circuits, software, and/or hardware that implement the functionality.
Processors suitable for the execution of a computer program include, for example, general and special purpose microprocessors, and any one or more processors of any kind of digital computer. In summary, a processor receives instructions and data from a read-only memory or a random memory or both. A basic component of a computer is a processor that executes instructions, and one or more memory devices that store instructions and data. In general, a computer can include data and/or transmission data that can be used to couple one or more mass storage devices (eg, magnetic, magneto-optical disks, or optical disks) for storing data.
Data transmission and instructions can also occur on a communication network. Information vehicles suitable for the implementation of computer program instructions and information include all types of non-volatile memory, including, for example, semiconductor memory devices. The information carrier can be, for example, an EPROM, an EEPROM, a flash memory device, a magnetic disk, an internal hard disk, a removable disk, a magnetic optical disk, a CD-ROM and/or a DVD-ROM disc. The processor and the memory can be supplemented and/or added by specific purpose logic circuitry.
In order to provide interaction with the user, the above techniques can be implemented on a computer having a display. The display can be, for example, a cathode ray tube (CRT) and/or a liquid crystal display (LCD) monitor. The interaction with the user can be, for example, displaying information to the user, and a keyboard and a pointing device (such as a mouse or trackball) whereby the user can provide input to the computer (eg, with a user interface component) interactive). Other types of devices can be used to provide interaction with a user. Other devices may provide feedback to the user (eg, visual feedback, sound feedback, or tactile feedback), for example, in any type of inductive feedback. The input from the user can be, for example, in any type of recipient, including voice, speech, and/or tactile input.
The above described techniques can be implemented in a distributed computing system including a back end component. The backend component can be, for example, a data server, a relay software component, and/or an application server. The above described techniques can be implemented in a distributed computing system including a front end assembly. The front end component can be, for example, a client computer having a graphical user interface, a web browser, whereby a user can interact with an example implementation and/or other graphical use of the transmitting device. Interface. The components of the system can be interconnected by any type or media digital data communication (eg, a network).
Examples of networks include a local area network (LAN), a wide area network (WAN), an internet, a wired network, a wireless network, a packetized network, and/or a carrier. network. Connections to the network may include, but are not limited to, long-range wireless connections, short-range wireless connections, and a variety of wired connections including, but not limited to, telephone lines, cable lines, power lines, and the like.
Packetized networks may include, for example, the Internet, a carrier Internet Protocol (IP) network (such as a regional network LAN), a wide area network (WAN), and a campus area network (CAN). Metropolitan Area Network (MAN), Home Area Network (HAN), a private IP network, an IP private branch exchange (IPBX), a wireless network ( For example, Radio Access Network (RAN), 802.11 network, 802.16 network, General Packet Radio Service (GPRS) network, HiperLAN, and/or other packetized networks. The circuit network may include, for example, a Public Switched Telephone Network (PSTN), a Private Branch Exchange (PBX), a wireless network (eg, RAN, Bluetooth, coded multi-directional proximity ( Code-Division Multiple Access (CDMA) network, Time Division Multiple Access (TDMA) network, Global system for mobile communications (GSM) network, and/or other circuit networks road.
In some examples, the server uses a protocol stack to transmit the encoded media bitstream. The stack may include, but is not limited to, Real Time Transport Protocol (RTP), User Data Block Agreement (UDP), and Internet Protocol (IP). When the protocol stack is oriented as a packet, the server encapsulates the encoded media bit stream into the packet. For example, when using RTP, the server encapsulates the encoded media bit stream into an RTP packet according to an RTP payload format. Basically, each media type has a proprietary RTP payload format.
In other examples, the server can be connected to a gateway via a communication network. The gateway can perform different kinds of functions, such as translating a packet stream stacked according to a communication protocol to another communication protocol stack, merging and distributing the data stream, and manipulating according to the downlink and/or receiver capabilities Data stream, for example, controlling the bit rate of the forward stream based on common downlink network conditions. Examples of gateways include MCUs, gateways between circuit-switched and packet-switched video phones, push-to-talk over cellular (PoC) servers, and digital video broadcast handhelds. An IP wrapper in a (Digital Video Broadcasting-Handheld, DVB-H) system, or a set-top box that can be broadcast locally to a home wireless network. When using RTP, the gateway is referred to as an RTP mixer or an RTP translator and is essentially an endpoint of an RTP connection.
The system can include a client and a server. A client and a server are roughly remote from each other and interact substantially via a communication network. The relationship between the client and the server is established by a computer program running on the respective computers and has a client-server relationship with each other.
The communication device may include, for example, a computer, a computer having a browser device, a telephone, an IP telephone, a mobile device (such as a mobile phone, a personal digital assistant (PDA) device, a laptop, an email device). And / or other communication devices. The browser device includes, for example, a computer (eg, a desktop computer, a laptop computer) having a global information web browser (eg, Microsoft provided by Microsoft Corporation) Internet Explorer , and Mozilla provided by Mozilla Firefox). The mobile computing device includes, for example, a number of person assistants (PDAs).
The communication device can be stationary or mobile when carried by, for example, a person in action. The communication devices may also be located in a transport mode including, but not limited to, automobiles, trucks, taxis, buses, trains, boats, airplanes, bicycles, locomotives, and the like. Some or all of the communication devices can transmit and receive calls and messages and communicate with the service provider via a wireless connection to a base station. The base station can be connected to a network server that allows communication between the mobile telephone network and the network. The system can include additional communication devices and different types of communication devices.
The communication devices can communicate using a variety of transmission techniques including, but not limited to, coded multi-directional proximity (CDMA), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), and time-sharing. Multi-directional proximity (TDMA), Frequency Division Multiple Access (FDMA), Transmission Control Protocol/Internet Protocol (Transmission Control Protocol/Internet Protocol, TCP/IP), Short Messaging Service (Short Messaging Service, SMS), Multimedia Messaging Service (MMS), Email, Instant Messaging Service (IMS), Bluetooth, IEEE 802.11, etc. A communication device embodied in various embodiments of the present invention can communicate using a variety of media including, but not limited to, radio, infrared, laser, cable connections, and the like.
The plural forms of inclusion, inclusion, and/or each are open, and include the listed parts, and may include additional parts not listed. And/or is an open end and includes one or more of the listed parts, and combinations of the listed parts.
The foregoing description of the specific embodiments has been presented for purposes of illustration and description. The above description is not intended to be exhaustive or to limit the embodiments of the invention to the precise embodiments disclosed. The selection and description of the specific embodiments discussed herein are illustrative of the principles and nature of the various embodiments of the invention, Can be applied to the specific use considered. The features of the specific embodiments described herein may be combined in a combination of all possible methods, devices, modules, systems, and computer program products.
In some embodiments, any of the aspects described below can include one or more of the following examples.
In one aspect, a method includes receiving a multimedia data block, the multimedia data block including a plurality of data portions, and inserting the plurality of data portions to form an inserted data block based on a priority order, the priority order Based on a multimedia material type associated with one of the data portions; and transmitting the inserted data block.
In other examples, the priority order is a reverse time recording order, a time recording order, a data portion size order, and/or a data type order.
In another aspect, an apparatus includes: a transceiver configured to receive a multimedia data block and to transmit an inserted data block, the multimedia data block including a plurality of data portions; and an inserter And configured to insert the plurality of data portions, the inserted data blocks are formed based on a priority order, the priority order being based on a multimedia material type of the data portions.
In another aspect, a computer readable storage medium encoded with instructions, when executed by a computer, performs: receiving a multimedia data block, the multimedia data block including a plurality of data portions; The plurality of data portions form an inserted data block based on a priority order based on a multimedia material type associated with one of the data portions; and transmitting the inserted data block.
In another aspect, a method includes: receiving one or more data portions of an inserted multimedia data block; based on time information associated with the one or more data portions and/or the one or more The data type information of the data portion determines the first data portion of one or more of the data portions; and the time information of the earliest data portion of one of the subsequent data blocks determines the time of the first data portion One-time project.
In another aspect, an apparatus includes: a transceiver configured to receive one or more data portions of an inserted multimedia data block; and a media processor configured to: correlate The time information of the one or more data portions and/or the data type information of the one or more data portions determines the first data portion of one or more of the data portions; and based on the subsequent data region The playback time information of the earliest data portion of the block determines a time-interval item in the first data portion.
In other examples, the device further includes the media processor, which is additionally configured to determine the play time information based on a segment received by one of the inserted multimedia data blocks.
In some examples, the device further includes the media processor configured to determine a time course in the first data portion based on play time information and/or freeze time information of an earliest data portion of the subsequent data block The time project.
In another aspect, a computer readable storage medium encoded with instructions that, when executed by a computer, perform: receiving one or more data portions of an inserted multimedia data block; The time information of the one or more data portions and/or the data type information of the one or more data portions determines the first data portion of one or more of the data portions; and based on a subsequent data region The playback time information of one of the earliest data portions of the block determines a one-time intermediate time-history item of the first data portion.
100. . . Communication system
110. . . Communication device
110a. . . Wireless phone
110b. . . Personal digital assistant
110c. . . computer
110z. . . TV
120. . . Communication network
130, A130a, B130b, Z130z. . . Multimedia server
200a, 200b. . . Communication system
210. . . Communication device
211. . . transceiver
212. . . decoder
213. . . Media processor
214. . . monitor
215. . . Storage device
230. . . Multimedia server
231. . . transceiver
232. . . Source encoder
233. . . Multiplexer
234. . . Forward error correction encoder
235. . . Inserter
236. . . Storage device
325, A325a, B325b, Z325z. . . Multimedia storage server
330. . . Multimedia server
331,A331a, B331b, Z331z. . . Source encoder
332, A332a, B332b, Z332z. . . Data stream
333. . . Multiplexer
334. . . Source block
335. . . Forward error correction encoder
336. . . Inserter
337a. . . Inserted source block
337b. . . Patching information
400. . . system
425, A425a, B425b, Z425z. . . Multimedia storage server
430. . . Multimedia server
431, A431a, B431b, Z431z. . . Source encoder
432, A432a, B432b, Z432z. . . Data stream
433. . . Multiplexer
434. . . Inserted source block
435. . . Forward error correction encoder
436. . . Inserter
437a. . . Coded source block
437b. . . Patching information
534a, 534b. . . Source block
536a, 536b. . . Inserter
537a, 537b. . . Inserted source block
600a, 600b. . . Scheduler
i 610a, i+1 610b, i+2 610c. . . Source block
614a. . . First section
616a. . . Second section
620. . . timeline
622. . . Transfer time
624. . . Minimum buffer time
626a, 626b. . . Early replay begins
628. . . Freezing period
710a, 710b. . . X axis
720a, 720b. . . Y-axis
730b. . . Transport stream
732a. . . Source block
734a. . . Source block
736a. . . Source block
810. . . Communication device
820. . . network
830. . . Multimedia server
851. . . antenna
852. . . shell
853. . . horn
854. . . monitor
855. . . keyboard
856. . . microphone
857. . . Storage
858. . . battery
864. . . Interactive display
864. . . Multimedia library
900a, 900b. . . Data block description
910a, 930a, 910b, 930b, 935b. . . General descriptor
920a, 920b, 925b. . . Insert descriptor
Specific embodiments are described with reference to the accompanying drawings, in which:
Figure 1 is an example communication system for data transfer;
2A and 2B are block diagrams showing an example communication device and a block diagram of an example server;
3 is a detailed block diagram of a server in accordance with an exemplary embodiment of the present invention;
4 is a detailed block diagram of a server in accordance with another exemplary embodiment of the present invention;
5A and 5B are block diagrams of example multimedia source blocks and the same respective multimedia source blocks after being inserted, in accordance with an exemplary embodiment of the present invention;
6A and 6B are timelines of an example scheduling program executed by a receiving communication device in accordance with an exemplary embodiment of the present invention;
FIG. 7A is a presentation/display time stamp of a data portion of a data block inserted according to an exemplary embodiment of the present invention; FIG.
Figure 7B is a presentation/display time stamp of the non-inserted data portion of the same data block as in Figure 7A;
Figure 8 is an example communication device;
Figures 9A and 9B are examples of data block descriptions including insertion descriptors;
10A is a flow chart of an example insertion procedure performed by a server in accordance with an exemplary embodiment of the present invention;
10B is a flow chart of an example insertion procedure performed by a server in accordance with an exemplary embodiment of the present invention; and
Figure 11 is a flow diagram of an exemplary scheduling and de-insertion procedure performed by a communication device in accordance with an exemplary embodiment of the present invention.
A method for interleaving a data block, comprising the steps of: receiving an insertion attribute, wherein the insertion attribute comprises a sequence of one or more data portions with a priority order and an inserted multimedia data block Correlation information, wherein the insertion attribute is communicated using a service description protocol attribute; receiving the one or more data portions of the inserted multimedia data block, wherein the insertion of the multimedia data block is based at least in part on The priority order, the priority order is based on type information associated with the one or more data portions; based on time information associated with the one or more data portions, the received insertion attribute, and the Or data type information of the plurality of data portions, determining a first data portion of one or more of the data portions; and correlating one or more of the data received based on the inserted multimedia data block The time information of the data part and the playing time information of the earliest data part of one of the next data blocks are determined to be in the first data part. When the process to start playing one point of time, playback time of the next data block in which the earliest part of the Department of Information Resources section based on a minimum buffer time decision.
The method of claim 1, wherein the method further comprises at least one of the following steps: configuring the one or more data portions based on at least one of: associated with the one or more data portions The time information, the data type information of the one or more data portions; playing one or more data portions of the configuration; and based on the time point determined to start playing in the time course, The first data portion is played, wherein the playing a data portion includes displaying the data portion and/or restoring the data portion.
The method of claim 1, wherein the inserted multimedia data block comprises one or more of the following: an audio data portion, a video data portion, an image data portion, a text data portion, and an animation material. section.
The method of claim 1, wherein the step of determining the time point to start playing in the time period comprises: determining the one or more materials of the inserted multimedia data block received Part of the playback duration.
The method of claim 1, wherein the method further comprises: determining, based on a difference between time information associated with the next multimedia material block and time information associated with the one or more data portions A freeze period.
The method of any one of claims 1 to 5, wherein the data type information of the one or more data portions comprises a multimedia type of one or more of the data portions.
An apparatus for inserting a data block, comprising: a processor; and a memory including a computer code, the memory and the computer code being combined to operate with the processor to cause the device to execute At least the following: receiving an insertion attribute, wherein the insertion attribute includes information associated with a priority order and a sequence of one or more data portions of an inserted multimedia material block, wherein the insertion attribute is a service description Receiving a protocol attribute; receiving the one or more data portions of the inserted multimedia data block, wherein the insertion of the multimedia data block is based at least in part on the priority order, the priority order being based on the one Or type information associated with the plurality of data portions; determining the time based on time information associated with the one or more data portions, the inserted insertion property, and data type information of the one or more data portions Waiting for the first data portion of one or more data portions; and receiving the multimedia data block for the insertion The amount of information, the time information associated with the one or more data portions, and the playback time information of one of the earliest data portions of the next data block, determining that the first data portion is to be played in a time interval At one time point, the play time information of the earliest data portion of the next data block is determined based on a minimum buffer time.
The device of claim 7, wherein the memory and the computer code are combined to operate with the processor, and the device is caused to perform at least one of the following steps: based on at least one of the following: Configuring the one or more data portions: the time information associated with the one or more data portions and the data type information of the one or more data portions; playing one or more materials of the configuration And playing the first data portion based on the determined time point to start playing in the time period, wherein playing a data portion includes displaying the data portion and/or restoring the data portion.
The device of claim 7, wherein the inserted multimedia data block comprises one or more of the following: an audio data portion, a video data portion, an image data portion, a text data portion, and an animation material. section.
The device of claim 7, wherein the memory is The computer code is configured to operate with the processor, and the device is further configured to determine the duration of the playback of the one or more portions of the received multimedia data block.
The device of claim 7, wherein the memory and the computer code are combined to operate with the processor, and the device is based on time information and association associated with the next multimedia data block. A freeze period is determined by the difference between the time information of the one or more data portions.
The device of any one of claims 7 to 11, wherein the data type information of the one or more data portions comprises a multimedia type of one or more of the data portions.
A non-transitory computer readable storage medium having encoded instructions that, when executed by a computer processor, causes a device to perform the method of claim 1 of the scope of the patent.
An apparatus for inserting a data block, comprising: a first receiving component, configured to receive an insertion attribute, wherein the insertion attribute comprises one or more data sections with a priority order and an inserted multimedia data block a sequence of associated information, wherein the insertion attribute is communicated using a service description protocol attribute; a second receiving element for receiving the one or more data portions of the inserted multimedia data block, The insertion of the multimedia data block is based at least in part on the priority order based on the type information associated with the one or more data portions; a first determining component for correlating with the Determining the first data portion of one or more of the one or more data portions by waiting for time information of one or more data portions, the insertion property received, and data type information of the one or more data portions; And a second determining component, configured to time information related to the one or more data portions based on a quantity of data received for the inserted multimedia data block The playback time information and data part of one of the earliest next data block to determine for the first data portion to start playing one point at a time process time, playing time of the next data block in which the earliest part of the information The information is based in part on a minimum buffer time.
A communication system comprising: a communication network; at least one multimedia server; and at least one communication device, wherein a multimedia server comprises: a processor; and a memory including a computer code, the memory and the computer code Composing to operate with the processor such that the multimedia server performs at least the following steps: receiving a multimedia data block, the multimedia data block including a plurality of data portions; inserting the plurality of data portions based at least in part on a priority order Forming the inserted data block, the priority order is based on type information associated with the plurality of data portions; generating an insertion attribute based on the information associated with the inserted data block, wherein the insertion attribute includes the priority Sequence-related information; transmitting the insertion attribute using a service description protocol attribute; and transmitting the inserted data block to the at least one communication device via the communication network, and wherein one of the communication devices comprises: a processor; and including a computer program The memory of the code, the memory and the computer code are grouped Operating in conjunction with the processor, the communication device performs at least the steps of: receiving an insertion attribute, wherein the insertion attribute includes a sequence associated with a priority order and one or more data portions of an inserted multimedia material block Information, wherein the insertion attribute is communicated using a service description protocol attribute; receiving the one or more data portions of the inserted multimedia data block; based on time information associated with the one or more data portions Determining, by the data of the one or more data portions, the first data portion of the one or more data portions; and based on the multimedia data block for the insertion The quantity of the received data, the time information associated with the one or more data portions, and the playing time information of one of the earliest data portions of the next data block, to determine that the first data portion is to be Start playing one of the time points, wherein the playback time information of the earliest data portion of the next data block is based in part on a minimum buffer time Decision.
TW098136942A 2008-10-30 2009-10-30 The method for inserting data block and equipment TWI549436B (en)
TW201023531A TW201023531A (en) 2010-06-16
TWI549436B true TWI549436B (en) 2016-09-11
TW098136942A TWI549436B (en) 2008-10-30 2009-10-30 The method for inserting data block and equipment
US9015564B2 (en) * 2009-08-19 2015-04-21 Qualcomm Incorporated Content delivery system with allocation of source data and repair data among HTTP servers
JP5058209B2 (en) * 2009-05-22 2012-10-24 株式会社日立製作所 Data processing system for reverse reproduction in stream data processing
EP2529528B1 (en) * 2010-01-28 2018-01-10 Thomson Licensing A method and apparatus for parsing a network abstraction-layer for reliable data communication
CN106330713A (en) * 2015-06-29 2017-01-11 华为技术有限公司 Message transmission method and device
CN106780349A (en) * 2016-11-21 2017-05-31 深圳开立生物医疗科技股份有限公司 Space compound imaging method and device based on FPGA
AU2006248710B2 (en) * 2005-05-19 2011-01-20 Nokia Corporation System and method for providing unequal error protection to priority labeled datagrams in a DVB-H transmission system
EP1974540A1 (en) * 2006-01-17 2008-10-01 Thomson Licensing Gateway for receiving digital television broadcasting services, terminal and corresponding methods
EP2351325A4 (en) 2013-01-02
EP2227017A1 (en) 2010-09-08 Media channel management
TWI572199B (en) 2017-02-21 Corresponding to the input from the adaptive stream transcoded content to output the content stream transmitting method and apparatus ilk
EP2486705A2 (en) 2012-08-15 Low latency cacheable media streaming