2. RMCP (Remote Music Control Protocol)

RMCP:

Network Protocol for Music Applications

Remote Music Control Protocol

Share symbolized musical information via networks
integrate MIDI and LAN (Ethernet)

Design and Applications

Ethernet

MIDI

Efficient information sharing among processes
connection-less protocol on UDP/IP
School of Science and Engineering, Waseda University, Japan

Masataka Goto

Ryo Neyama

support broadcast-based information sharing
without the overhead of multiple transmission

Yoichi Muraoka

MIDI messages
note onnote off

RMCP packetLAN1997/09/27 ICMC97

No.1

No.5

1. Introduction

include various musical information

Multi-server Multi-client Model
RMCP server

Our Goal
RMCP clients

Integrate MIDI and computer networks

RMCP client

RMCP server

RMCP client

generate RMCP packets

Internetbroadcast them to all RMCP servers

Network musical applications

connection-less one-directional transmission

live MIDI broadcasts
remote sessions via network

RMCP server

B

RMCP servers

A

receive all the broadcast RMCP packets

Computer supported sessions

utilize them in various ways

human-computer improvisation

sound output / music visualization / music-driven CG

interactive-graphics displays
No.2

No.6

add various servers without any extra packets

Design Policy

Network Protocol for Musical Information
Transmit symbolized musical information via network

Implement necessary functions
as small different processes
so that they can be reusable

Network musical applications
Distributed implementation of music-related softwareachieve good load-balancing

Easy system implementation and expansion

exploit various facilities
connected with different computerseach process can be devoted to its own small function
new function is achieved just by adding a new server

Efficient information sharing
over network is important

Good load-balancing

NetworkNo.3

No.7

Related Workeasy to allocate RMCP servers and clientson distributed computers

Basic RMCP Servers and Clients
RMCP clients

MIDI-based network protocols
connection-oriented
not emphasize low-latency information sharing
among multiple distributed processes

MIDI

RMCP MIDI Station
RMCP SMF Player

RMCP MIDI Receiver

MIDI

Ethernet

weak in efficient information sharing among devices
RMCP Animation Server

RMCP Sound Server

low bandwidth / just for local communication

MIDI

RMCP Display Server

Non-MIDI-based music protocols
generally presuppose the use of special devices
No.4

RMCP servers
No.83. Implementation of RMCP

4. Applications

2 bytes (16 bits)
RMCP magic number

flag

Networked Session

server number

header length
type of message
message lengthclient/user identification (R)client/user identification (G)optionalclient/user identification (B)type of time stamp
reserved
time stamp (sec)time stamp (msec)

Basic message
types

RMCP Header

RMCP Packet

Several players perform music together via Ethernet
listen to other players’ performances
see visualized performances
RMCP Display Server

RMCP Message Body

MIDI information:transmitting MIDI messages

beat information:beat synchronization

chord information:

chord name and voicing

RMCP Display Server

SMFABLAN

animation information: controlling computer graphics
No.9

RMCP MIDI Stationor
RMCP MIDI Receiver

RMCP Sound Server

RMCP SMF Player

No.13

Time Scheduling Using Time Stamps

RMCP MIDI Stationor
RMCP MIDI Receiver

Networked Session

Two kinds of RMCP packets
packet with time stamp
received before time stamp
received after time stamp
packet without time stamp

processed on timeprocessed immediately
processed immediately

RMCP Time Synchronization Server
enable RMCP servers to handle time stamps
as if all the internal clocks were synchronized
Offset Table
133.9.68.56 0
133.9.68.57 -50
133.9.68.58 +20
Offset Table
133.9.68.56 +50
133.9.68.57 0
133.9.68.58 +70

IP Address:
133.9.68.57
Host B

1250

No.10

IP Address:
133.9.68.56

1200

Host A+50

1000

Offset Table
133.9.68.56 -20
133.9.68.57 -70
133.9.68.58 0

IP Address:
133.9.68.58-201180

1050

Host C980

No.14

RMCP over the Internet

Improvision

RMCP Gateway

Musical-instrument interface for untrained novices

bidirectional relay of RMCP packets over the Internet

improvise unconventional music easily
by clicking and dragging a computer mouseconnection-oriented protocol on TCP/IP

relay with thespecified latency

1350

Host BRMCP Gateway
Host A

1050

RemoteGIG

1300-501000+2000+500-20
Host E

Host D3780

3800

connection

live MIDI broadcasts

LAN

RMCP client

RMCP Gateway

RMCP server

1500

1480

+2000 specified latenc
y for relay
WAN+/- 500 internal clock
difference compensation

RMCP servers and clients can communicate
as if different LANs were the same network

RMCP Improvision Display ServerImprovision

ALAN
RMCP Improvision

Improvision

RMCP programming libraries in both C and Java
tested on various computers and OSs
Solaris-2.5
Windows-95

SunOS-4.1.3
Windows-NT

Communication delay between server and client
Ave: 0.30 ms
SD: 0.06 ms

Min: 0.28 ms
Max: 1.24 ms
note on

10Mbps Ethernet

clientLANserver
SGI Indigo2 (IRIX-5.3)

No.12

Fast enough compared with MIDI (31.25 Kbps)

No.16

novisio
Impr
C

RMCP Sound Server

Experimental Results

direction: pitch
speed:
volumeB

RMCP Improvision

No.15

IRIX-6.2
Linux-2.0CImpr
ovision

ALAN

No.11

IRIX-5.3
HP-UX

B

RMCP Improvision

VirJa Session

RemoteGIG

Virtual jazz session system

BA

Remote session with delay

multimodal interaction among all players

impossible to avoid network latency over the Internetsounds / 3D computer graphics / gesturesmodels of traditional sessions are not usefulPlayer 1: Pianist

Player 1: Pianist

Human

Human

ComputerComputer

RMCP MIDI Receiver

RMCP Camera Analyzer

raRerameal
CCa

ComputerEthernetalameRe

Human

Player 3: Drummer
Human
Computer

Virtual Camera

Player 2: Bassist

Player 3: Drummer

Human

Virtual Camera

Real-time CG

overcome long network latency

Human
ComputerComputer

RMCP Player Server

RMCP Player Server

RMCP Beat Provider

RMCP Beat Provider

RMCP Animation Server

Real-time CG

PianistInnovative remote session over the Internet

Virtual Jazz Trio
Player 2: Bassist

RMCP gateways provide a certain constant latency
with very small deviation

RMCP Sound Server

Bassistoffer a new possibility for future remote sessions

RMCP Animation Server

Drummer

No.17

No.21

VirJa Session

1 period

RemoteGIG

1 period

1 period

time

Site A
Player A A-1

A-2

A-3

B-1

B-2

A-1

A-2

B-2

B-3

Site B

Assumption

Player B B-1

tempo: constantchord progression: repetitive(ex. 12-bar blues)

playing background drums to keep the tempo

Turn the network latency to its advantage
other players’ performances are delayed
for the constant period of chord progression
No.18

No.22

5. Conclusion

Virtual Dancer ‘‘Cindy’’
Interactive performance of a music-controlled CG dancer

Summary

two players choreograph Cindy by their improvisationinteract through music and 3D computer animation

MIDI Instrument

RMCP: Network protocol for music applications
Multiple distributed processes can sharesymbolized musical information such as MIDICG Dancer

MIDICG Dancer
SoundRMCP Sound Server

RMCP Animation ServerFeatures:

Sound

Interactive PerformanceCG

RMCP Music Analyzer

efficient broadcast-based information sharing over LANCGRMCP MIDI Receiver

RMCP MIDI Receivertime-scheduling using time stamps

Sound
MIDISound

Player A
Drummer

No.19

delayed performances can fit the chords

Player B
Guitarist
PianistVirtual Dancer ‘‘Cindy’’

Player A
Drums

live MIDI transmission over WAN such as Internet

MIDI

Player B
PianoGuitar

No.23

Utilized for various applications

Future Work
Provide an API ‘‘Jam’’ to make
RMCP and MIDI usable in Java applets
Implement various RMCP-based applications

RMCP Software Package Distribution
http://www.info.waseda.ac.jp/muraoka/members/goto/RMCP/
basic RMCP server and client programs
RMCP programming libraries
No.20

No.24