Digital video/audio capture device and method thereof

In audio/video (AV) capture, two video streaming adapters (VSAs) receive an original input video signal and a compressed input video signal respectively to generate an uncompressed video streaming and a compressed video streaming respectively. The uncompressed video streaming and the compressed video streaming are transmitted to an external device by way of time division duplex. An interlock mechanism prevents concurrent transmission of the uncompressed video streaming and the compressed video streaming, so that the loss of real-time video streaming is reduced.

This application claims the benefit of Taiwan application Serial No. 98102072, filed Jan. 20, 2009, the subject matter of which is incorporated herein by reference.

BACKGROUND

The application relates in general to a digital audio/video (AV) capture device and a method thereof, and more particularly to a digital AV capture device and a method thereof, capable of outputting an uncompressed video streaming and a compressed video streaming.

2. Description of Background

Current USB video device class (UVC) audio/video products can be used in Windows XP SP2 and later operating systems or Vista system in a plug-and-play manner without installing any driver. Examples of the UVC AV products include digital webcam, digital camera, analog image converter, TV stick and etc. Webcam, which is one of earliest AV products that support UVC protocol, is widely used and has a large demand in the market.

As the built-in digital webcam has become a standard equipment of a notebook computer, more application of the built-in digital webcam are developed. For example, notebook or personal computer with a built-in digital webcam can be used in digital home or video conference. For this kind of application, video streaming is indispensable. Let video conference be taken for example. Normally, the video streaming has two formats: compressed signal format (such as MPEG) for transmission on the network, and uncompressed signal format (such as YUV format) for playing at local terminal.

FIG. 1shows a conference frame displayed on a display unit (such as the screen of a notebook or a personal computer) during video conference. As inFIG. 1, during video conference, both a local terminal frame110and a remote terminal frame120are shown on the display unit. The local terminal frame110is a uncompressed video streaming outputted from the built-in digital webcam and played on the display unit of the local terminal computer. The remote terminal frame120is a compressed signal outputted from the remote terminal, de-compressed by the local terminal computer and played on the display unit of the local terminal computer.

The conventional digital webcam is disadvantaged by supporting one video format transmission only. For example, all the video streamings outputted from the conventional digital webcam are uncompressed. Therefore, during video conference, the local terminal computer performs compression/decompression, such as, compressing the uncompressed video streaming outputted from the digital webcam and transmitting to the remote terminal computer through a network, and decompressing and playing the compressed signal transmitted from the remote terminal computer. However, the above operation will make the local computer overloaded with compression/decompression.

Thus, example of the invention provide new digital AV capture configuration. The conventional configuration which supports only one flow of UVC real-time isochronous transfer is improved to be capable of supporting two flows of isochronous transfer (that is, capable of transmitting both a compressed video streaming and an uncompressed video stream) by one USB digital AV capture element only (such as only one USB digital webcam). The local terminal computer concurrently receives the video streaming in two formats, so the load of performing compression and decompression by the local terminal computer is reduced.

Moreover, the new USB digital AV capture configuration according to examples of the invention automatically adds a UVC compliant payload header to the video streaming, hence making the implementation in video conference or digital home easier.

SUMMARY OF THE INVENTION

Examples of the invention are directed to an audio/video (AV) capture device and a method thereof. According to the concept of time division duplex, a compressed video streaming and an uncompressed video streaming can be outputted in time division duplex by only one universal serial bus (USB) digital AV capture element. The local terminal computer concurrently receives two formats of video streaming, so the load of performing compression and decompression by the local terminal computer is reduced.

Examples of the invention are directed to an AV capture device and method thereof. As the USB video class (UVC) payload header can be automatically added, the load of the micro controller is relieved and loss of real-time data is reduced. Thus, the implementation of video conference or digital home is made easier.

According to a first example of the present invention, an AV capture device is provided. The AV capture device includes a signal source, an AV processor, a first video streaming adapter (VSA), a second video streaming adapter (VSA), a micro controller, and a transmission unit. The signal source captures an external image to generate a first input video signal. The AV processor receives and compresses the first input video signal generated by the signal source to generate a second input video signal. The first VSA receives the first input video signal generated by the signal source and adds a first payload header onto the first input video signal to generate a first video streaming. The second VSA receives the second input video signal generated by the AV processor and adds a second payload header onto the second input video signal to generate a second video streaming. The micro controller generates a command to the first VSA and the second VSA. The transmission unit, coupled to the first VSA and the second VSA, transmits the first video streaming generated by the first VSA and the second video streaming generated by the second VSA to an external device by time division duplex. Wherein, the first VSA and the second VSA communicate with each other through an interlock mechanism to avoid the first VSA and the second VSA concurrently outputting the first and the second video streaming to the transmission unit.

According to a second example of the present invention, a digital AV capture method is provided. An external image is captured to generate a first input video signal. The first input video signal is received and compressed to generate a second input video signal. The first input video signal is received and a first payload header is added onto the first input video signal to generate a first video streaming. The second input video signal is received and a second payload header is added onto the second input video signal to generate a second video streaming. The first video streaming and the second video streaming are transmitted to an external device by time division duplex. An interlock mechanism is performed during time division duplexing transmission to avoid the first and the second video streaming being concurrently transmitted to the external device.

Examples of the invention will become apparent from the following detailed description of the non-limiting embodiments. The following description is made with reference to the accompanying drawings.

DETAILED DESCRIPTION OF EXAMPLES OF THE INVENTION

According to the new USB digital AV capture configuration provided in the embodiments of the invention, the isochronous transfer of two formats of video streaming (that is, a compressed video streaming and an uncompressed video streaming are concurrently transmitted) is supported by time division duplex. The local terminal computer concurrently receives two formats of video streaming, so the load of performing compression and decompression by the local terminal computer is reduced.

According to the new USB digital AV capture configuration provided in the following embodiments of the invention, a UVC compliant payload header is automatically added, hence making the implementation of video conference or digital home easier.

Here below, a video streaming includes video and/or audio data.FIG. 2A˜FIG.2C show a micro frame in a video streaming, respectively. As indicated inFIG. 2A˜FIG.2C, a micro frame includes a start of frame (SOF)210, a payload header220and a plurality of real-time data packets231˜233. In each micro frame, the payload header220must be updated and is located at the start of the first real-time data packet. Generally speaking, a UVC payload header includes the following information such as whether the streaming belongs to the same video frame, whether the streaming is in a photo mode, whether the streaming is an end of video frame, and the total time length of the video streaming. The real-time data packets231˜233include AV data.

Conventionally, the payload header is filled in by a micro controller inside a digital webcam internal. In a USB micro frame, normally the micro controller has to complete the following operations: 1. Collect the information required in the payload header such as timestamp and so on. 2. Copy the AV data of the current micro frame to the memory in which the payload header is stored, wherein the storage position of the AV data must be after that of the payload header. 3. Control the digital webcam to sequentially output the payload header and the AV data. 4. Confirm that the digital webcam has completed output of data.

The above operations must be repeated once within each micro frame, that is, within every 125 microsecond (125 μsecond). A micro controller may not be able to complete the above operations within the above time limit if the processing speed of the micro controller is too slow or the micro controller has other tasks to perform at the same time. If the above operations cannot be finished within the time limit, the AV data may be lost and the real-time AV transmission cannot be achieved.

Therefore, in the hardware configuration provided in the embodiment of the invention, a part of the payload header information is automatically updated or retrieved by other element than the micro controller, and during transmitting a micro frame, other element than the micro controller outputs the payload header first and the AV data next. The micro controller largely reduces the job required for real-time streaming transmission, not only making the system more efficiently but also avoiding the loss of video data.

Referring toFIG. 3, a block diagram of a digital AV capture device capable of supporting USB and UVC according to an embodiment of the invention is shown. As indicated inFIG. 3, the digital AV capture device300includes a signal source310, an AV processor315, two video streaming adapters (VSA)320and325, an advanced high-performance bridge (AHB)330, a micro controller335, a data bus340, a command bus345, a USB device controller350and a USB physical layer355. The USB device controller350at least includes a direct memory access (DMA)351.

The digital AV capture device300supports two flows of USB and UVC compliant video streaming, but only one USB device controller is required for transmitting the two flows of video streaming to the PC360at back stage. Furthermore, the bandwidth of the two flows of video streaming can be dynamically changed so that the maximum bandwidth of USB isochronous transfer can achieve 24.576 MB/sec. The digital AV capture device300automatically adds a UVC compliant payload header.

The photo-sensing element inside the signal source310captures an external image to generate a video signal which enters both the VSA320and the VSA325respectively. The flow of video signal that enters the VSA320is uncompressed video signal, and the other flow of video signal is processed by the AV processor315and then enters the VSA325. For example, the signal is compressed as H.264 format by the AV processor315.

The AV processor315compresses the video signal generated by the signal source310and further transmits the compressed video signal to the VSA325.

When the VSA320receives the uncompressed video signal and the VSA325receives the compressed video signal, the VSAs320and325encode the received video signals as video streamings. Also, the VSAs320and3255, according to the parameter from the micro controller335, automatically add a UVC compliant payload header and automatically set the DMA351of the USB device controller350through the data bus340so as to transmit the two flows of video streaming to the PC360by time division duplex.

In the present embodiment of the invention, only one of the VSAs320and325at a time can transmit data to the PC360through the USB device controller350. Thus, by way of an interlock mechanism and an interlock signal IL, only one of the VSAs320and325at a time can transmit data. That is, the VSA325has to wait when the VSA320transmits a part of uncompressed video streaming to the PC360in one micro frame. To the contrary, the VSA320has to wait when the VSA325transmits a part of compressed video streaming to the PC360in one micro frame.

The uncompressed video streaming transmitted to the PC360from the VSA320can be played on the PC360directly, and the compressed video streaming transmitted to the PC360from the VSA325can be further transmitted from the PC360to be sent and played on a remote terminal computer.

The AHB330serves as a bridge between two buses having different transfer rates. The micro controller335writes command on the command bus345and sets the USB device controller350through the AHB330so as to support UVC isochronous data transfer.

The command bus345is used for transmitting a command, a parameter, an instruction, and so on. The data bus340is used for transmitting AV data. Thus, the USB real-time video streaming that require high speed transmission and the command that does not require high speed transmission can be separated and transmitted through two different internal buses, not only assuring the accuracy of the video streaming but also meeting the effectiveness of the micro controller335in processing the commands.

The USB device controller350is used for transmitting the uncompressed video streaming transmitted from the VSA320and the compressed video streaming transmitted from the VSA325to the PC360by way of time division duplex. Moreover, after the USB device controller350transmits one data packet, the USB device controller350will inform the VSA320or325, retrieve a next packet from the VSA320or325and further transmit the next packet to the DMA351. That is, the VSA320or325transmits one data packet (and its payload header if the data packet is the first one in the micro frame) at a time to the DMA351.

The USB physical layer355is an interface between the USB device controller350and the PC360.

Next, referring toFIG. 4, the VSA configuration according to the embodiment of the invention is shown. In the present embodiment of the invention, the VSAs320and325are the same or similar and are exemplified by the VSA320here.

As indicated inFIG. 4, the VSA320includes a multiplexer (MUX)410, a counter415, a controller420, a DMA setting unit425, a packet buffer430, a header buffer435, a parameter buffer440, a USB event control unit445, an MUX450, an AHB slave device455, an AHB slave device460and an AHB master device465.

The counter415is used for counting data amount passing through the multiplexer410. According to the parameter transmitted from the micro controller335and then stored in the parameter buffer440, when the counter415counts that the data amount of the multiplexer410is equal to one packet size, the counter415packs the data as a packet and further stores the packet in the packet buffer430.

When packing the data as a packet, the counter415informs the controller420to start to encode a part of the UVC compliant payload header and store it in the header buffer435. The part of the UVC compliant payload header after encoding includes the following information such as whether the stream belongs to the same video frame, whether the stream is in a photo mode, whether the stream is an end of video frame and etc. Next, the counter415arranges the sequence of the real-time data packet as indicated inFIG. 2A˜FIG.2C.

When at least one packet is temporarily stored in packet buffer430, the DMA setting unit425of the VSA320informs the DMA351to transmit data. However, whether the data bus340and the DMA351are currently used by the other VSA must be checked through an interlock mechanism before the VSA uses the DMA351to transmit data. If the data bus340and the DMA351are currently used by the other VSA, then the VSA cannot transmit data until the other VSA has completed the transmission of a micro frame and released the control of bus usage. The operation of the interlock mechanism is disclosed below.

The USB event control unit445detects whether the transmission of each real-time data packet is completed. For example, when the DMA351completes the transmission of a real-time data packet, the USB device controller350sends a pulse signal back to the USB event control unit445through the data bus340. After receiving the pulse signal, the USB event control unit445informs the DMA setting unit425, the DMA setting unit425communicates with the DMA351, so that a next packet stored in the packet buffer430is transmitted to the DMA351through the multiplexer450, the AHB slave device455and the data bus340.

The multiplexer450, according to the control signal from other elements, determines whether to output the data packet or the payload header, so that the output sequence of the payload header and the data packet is conformed toFIG. 2A˜FIG.2C.

After the setting of the AHB master device465is completed, the DMA351of the USB device controller350retrieves a real-time data packet stored in the packet buffer430through the AHB slave device455. During the retrieval of data, the controller420, according to the format of the current video streaming, adds another part of the UVC compliant payload header payload header. For example, a time length of the video streaming is added. Thus, the transmission of the video streaming is completed.

The DMA setting unit425communicates with the DMA351through the AHB master device465and further sets the DMA351. Referring toFIG. 5, how the DMA setting unit425sets the DMA351is illustrated. Further, the interlock mechanism is performed by the DMA setting unit425.

The AHB slave devices455and460and the AHB master device465are interfaces between the VSAs and other elements. Particularly, the AHB slave device455is an interface between the VSA and the DMA351, the AHB slave device460is an interface between the VSA and the micro controller335, and the AHB master device465is an interface between the VSA and the DMA351.

InFIG. 4, the VSA320receives the uncompressed video streaming transmitted from the signal source310; and the VSA325receives the compressed video streaming transmitted from the AV processor315.

In the present embodiment of the invention, the AV capture device has two VSAs but only one USB device controller350. Thus, during the setting of the DMA351, the interlock mechanism is employed to avoid interrupt of the transmission of the isochronous packet by the VSAs320and325and avoid the VSAs320and325using the USB device controller350at the same time (that is, in the manner of time division duplex transfer) if the USB bandwidth is sufficient. The operations of the interlock mechanism according to the embodiment of the invention are illustrated inFIG. 5. The VSA320is used as an exemplification below. However, the operations of the VSA325are the same.

At the beginning, the DMA setting unit425is at an idle state510A. When receiving the USB transmission command, the DMA setting unit425enters a DMA confirmation state520A to check whether the DMA is busy.

If the DMA is busy (this implies that the other VSA) is transmitting data through the DMA351), then the DMA setting unit425returns to the idle state510A. If the DMA351is ready (not busy), then the DMA setting unit425enters into a write state530A.

In the write state530A, the DMA setting unit425writes some relevant parameters (such as the size, the number and the storage address) of the packet to the DMA351.

After that, the DMA setting unit425enters into an enablement state540A to activate the DMA351for retrieving the data packet stored in the packet buffer430and transmitting the retrieved data packet to the PC360. On entering into the write state530A, the DMA the setting unit425sends an interlock signal IL to the VSA325, so that the DMA setting unit of the VSA325maintains at the idle state and cannot use the DMA351to transmit data, hence avoiding loss of the real-time video streaming caused by the concurrence of two flows of video streaming. After the transmission of a micro frame is completed, the DMA the setting unit425enters into the idle state510A from the enablement state540A, so the control of the data bus340is passed to the other VSA. That is, the VSA325cannot transmit a compressed micro frame before the VSA320has completed the transmission of an uncompressed micro frame. The states510B˜540B are similar to the states510A˜540A, and the details are not repeated here.

According to the USB 2.0 high speed protocol, 8000 micro frames are transmitted per second, up to three real-time data packets are included in one micro frame, and the maximum size of each real-time data packet is 1024 bytes. Thus, the maximum bandwidth of each isochronous endpoint is 24.576 MB/sec. In USB protocol, under transmission bandwidth of 24.576 MB/sec, up to two isochronous endpoints are supported; that is, the sum of the maximum bandwidth of two isochronous endpoints is 49.152 MB/s.

The embodiment of the invention has three use modes as indicated in the table below. In the first use mode, only the uncompressed video streaming (YUYV format) is transmitted, and the maximum bandwidth is 24.576 MB/sec. This speed is sufficient for supporting the VGA (640×480) up to more than 30 video frames per second, and the bandwidth is about 18.423 MB/sec.

In the second use mode, only the compressed video streaming is transmitted, and the maximum bandwidth is 24.576 MB/sec too. As data is transmitted as compressed video streaming, it is costwise to adopt an 8 MB/sec bandwidth. Let the H.264 encoding format be taken for example. The 8 MB/sec bandwidth can support the Full-HD (1920×1080) up to 30 video frames per second, and the bandwidth is about 7.776 MB/sec if the compression rate is 12. However, the bandwidth can be dynamically adjusted to fit the needs in actual applications.

In the third use mode, the two VSAs (such as VSA320and VSA325) are both used; and under time division duplex, the maximum bandwidths are 24.576 MB/sec and 24.576 MB/sec, respectively. Likewise, as data is transmitted as compressed video streaming, it is costwise to adopt an 8 MB/sec bandwidth to transmit the compressed data transmitted from the VSA325.

In the above embodiments of the invention, the conventional configuration of supporting one flow of UVC isochronous transfer is improved to be capable of supporting two flows of isochronous transfer by using one USB device only. Beside, in the present embodiment of the invention, a UVC compliant payload header is automatically added by the VSA instead of a micro controller, hence reducing the load of the micro controller and avoiding the loss of the real-time data. Further, as the configuration of the present embodiment of the invention supports transmission of video streaming or audio/video streaming in two different formats, there is no need to install a driver in the application of a video conference or digital home, further reducing the burden in hardware requirements and increasing convenience in use.

It will be appreciated by those skilled in the art that changes could be made to the disclosed embodiments described above without departing from the broad inventive concept thereof. It is understood, therefore, that the disclosed embodiments are not limited to the particular examples disclosed, but is intended to cover modifications within the spirit and scope of the disclosed embodiments as defined by the claims that follow.