# Presentation: 508354

## High Quality 

**RGB/NTSC Video:**
**Improving Video Quality in the Access Grid**

- Matthew Delco, Andrew Iskandar, Lawrence Rowe
- Berkeley Multimedia Research Center
- University of California, Berkeley

## Berkeley Video Projects

- RTPtv
- Transmits “broadcast quality” television across an IP-based network (e.g., Internet2) using RTP
- Synchronized audio and video
- Full frame/full quality video
- High quality, RED-encoded audio
- RGB capture
- Transmits high-quality, low-loss RGB images
- Eliminates the use of “scan conversion”

- Access Grid Retreat 2002

## RTPtv

- Matthew Delco
- Berkeley Multimedia Research Center
- University of California, Berkeley

- Access Grid Retreat 2002

**Notes:**
High-level picture
2 pictures
low level stuff
arch diagram (network diagram)
hardware page

## RTPtv Technical Overview

- MJPEG (“motion JPEG”) codec
- NTSC or PAL, D1 or CIF size
- Video encoded/decoded using MJPEG board
- Constant bitrate mode: variable quality
- D1 interlaced: 5-30Mbps (10-15Mbps typical)
- CIF progressive: 1.5-10Mbps (5Mbps typical)
- Variable bitrate mode: fixed image quality
- Bitrate generally ~10Mbps, but can be 3-13Mbps
- Reduced frame rate can be used
- Another method: send only every other field

- Access Grid Retreat 2002

## RTPtv using Television

- Access Grid Retreat 2002

## RTPtv using PC Monitor

- Access Grid Retreat 2002

## D1 Interlaced Video

- Television video frame consists of two fields
- Rows of the two fields are combined/ interleaved to form a single video frame
- The first field is drawn, followed by the second

- Access Grid Retreat 2002

## CIF Progressive Video

- Transmit image that is 25% original size
- Only one field is transmitted
- Field is reduced horizontally 50%
- “Pixel Doubling” used to obtain a full-size image

- Field

- Field Shrunk Horizontally

- Restored Image on Receiver

- Access Grid Retreat 2002

## Transmitting JPEG over IP

- To transmit JPEG over IP, image divided into pieces that can be sent over the network
- Problem: vic drops entire frame if a single network packet is lost
- Solution: use “restart markers”
- RTPtv substitutes lost data with corresponding video image from previous field
- Open Mash vic to support partial frames (soon)
- Other vics can’t handle restart markers
- RTP processing & JPEG decoder are non-compliant

- Access Grid Retreat 2002

## RTPtv Hardware

- Video is encoded/decoded using MJPEG board
- LML33 produced by Linux Media Labs ($410)
- up to four LML33 boards in a single PC
- LML33 Linux driver is available via GPL
- RTPtv video currently operates only on a Linux PC
- Driver enhanced to provide precise timestamps
- Software decoding of video is possible, but lower quality
- Open Mash vic
- Can receive video up to 30fps for CIF
- For D1, takes one field and line-doubles image

- Access Grid Retreat 2002

## RTPtv Software

- RTPtv Software available in various forms
- Separate command-line applications/daemons
- Tcl/Tk GUI interface is available
- RPC interface (Tcl/DP syntax)
- Open Mash support for RTPtv/LML33
- vic can receive and software decode the images
- Support for restart markers is being refined
- vic doesn’t currently utilize the LML33 hardware
- Bob Riddle (Internet2) is working on this

- Access Grid Retreat 2002

## RTPtv/AG Integration

- Works today in Open Mash with Software Decoding
- Requires significant CPU resources
- P3/800 renders D1 @ 24 fps, CIF @ 30 fps
- LML33 Works in AG Capture Machine (Linux)
- Can use for raw or compressed capture
- Machine can also be used for receive/playback
- Hardware needed to scale to more streams
- Issues with incorporating hardware with AG

- Access Grid Retreat 2002

## RTPtv/AG Integration

- Scenario #1: Present Situation
- Problem: Restricted allocation of projectors

- AG

- RTPtv

- RGB

- NTSC

- Projectors

- Access Grid Retreat 2002

## RTPtv/AG Integration

- Scenario #2: AG Support Hardware Decoders
- Problem: Window GUI System Support
- Need new hardware / software support? (Marvel G400)

- Frame Buffer

- Hardware

- Hardware

- DMA

- DMA

- Access Grid Retreat 2002

## RTPtv/AG Integration

- Scenario #3: Introduce additional copies

- Frame Buffer

- Hardware

- Hardware

- RAM

- DMA

- DMA

- BitBlt

- BitBlt

- Access Grid Retreat 2002

## RTPtv/AG Integration

- Hardware
- Hardware needs to be smarter
- Multimon support is doubtful
- DMAs / PCI bus can cause issues
- Dual bus, or faster bus, would help
- Software
- “multicast tools” are not multi-threaded
- CPU scheduler introduces jitter in rendering
- MPEG
- Introduces latency
- Decoders rely on hardware too

- Access Grid Retreat 2002

## RGB Video

- Andrew Iskandar
- Berkeley Multimedia Research Center
- University of California, Berkeley

- Access Grid Retreat 2002

**Notes:**
High-level picture
2 pictures
low level stuff
arch diagram (network diagram)
hardware page

## RGB Technical Overview

- PCs use RGB video (not equiv. to NTSC RGB)
- Video capture cards only support NTSC/PAL
- Solution: scan-convert RGB to NTSC or PAL
- Problem: scan-conversion degrades quality
- Alt. solution: read out of PC’s RGB frame buffer
- Open Mash vic supports UNIX & MS Windows
- Problem: requires software to run on local machine
- PC owner may object to software installation
- Software may not work on PC (e.g., OS X)

- Access Grid Retreat 2002

## RGB Technical Overview

- Alt. solution: Multicast PowerPoint
- Problem: Product specific; not supported by MS
- Alt. solution: download ppt, pdf, ps file
- Problem: Viewer need to advance slides
- Alt. solution: find/make a RGB capture card

- Access Grid Retreat 2002

## RGB Capture Card

- Datapath (Derby, UK)
- VisionRGB Capture Card ($1k)
- Can capture 2 screens simultaneously
- Driver / SDK available for MS Windows
- 640x480, 800x600, 1024x768 (66Mpixel/sec)
- Image captured as 555, 565, or 888 pixels
- up to 20 frames/second
- Wire requirements:
- RGB plus HSync and VSync
- RGB with Sync on Green
- VisionRGB-PRO card supports 1600x1200 (280Mp/sec) & “RGB with Composite Sync”

- Access Grid Retreat 2002

## RGB Capture Software

- Open Mash allows RGB capture on MS Windows
- RGB capture handled the same as NTSC capture
- Since images are large, JPEG codec is used
- Future codec options: raw pixels, RLE, etc.
- Typically transmit 1 frame per second
- OM making improvements to JPEG software
- DCT cache for improved encode performance
- Restart markers / partial frames
- Some issues to be resolved
- Software buggy, signal auto-detect, 5-pin RGB

- Access Grid Retreat 2002

## Web Links

- RTPtv
- Web page:
- http://bmrc.berkeley.edu/~delco/rtptv/
- Technical report:
- http://bmrc.berkeley.edu/papers/2001/161/
- Open Mash:
- http://www.openmash.org/
- Linux Media Labs (JPEG hardware) web site:
- http://www.linuxmedialabs.com/
- Datapath (RGB hardware) web site:
- http://www.datapath.co.uk

- Access Grid Retreat 2002

## High Quality 

**RGB/NTSC Video:**
**Improving Video Quality in the Access Grid**

- Matthew Delco, Andrew Iskandar, Lawrence Rowe
- Berkeley Multimedia Research Center
- University of California, Berkeley

- Access Grid Retreat 2002

**Notes:**
High-level picture
2 pictures
low level stuff
arch diagram (network diagram)
hardware page