Continuous Media in Discrete Objects : Support for Collaborative Multimedia ? John Riedl Vahid Mashayekhi Computer Science Department University of Minnesota Minneapolis , MN -LRB- 612 -RRB- -624 -7372 September 4 , 1991 ? We gratefully acknowledge the support of the National Science Foundation -LRB- grant number NSF = IRI ? 9015442 -RRB- and the research funds of the Graduate School of the University of Minnesota . Continuous Media in Discrete Objects : Support for Collaborative Multimedia Abstract Multimedia has the same potential to revolutionize human-computer interfaces that bitmapped workstations have realized over the last decade . Achieving this potential requires the development of programming environments with integrated support for multimedia . SuiteSound is one such environment . SuiteSound is built in the Suite object-based system on a conventional UNIX operating system . SuiteSound objects incorporate multimedia by creating flows , which are streams of multimedia data moving through a sequence of objects . Flows bridge the gap between objects representing the state of an entity at a discrete point in time and space and continuous media such as live audio or video . This paper describes the design and implementation of SuiteSound on the Sun SparcStation . Three SuiteSound applications are described in detail : a teleconferencing tool , a voice annotator , and a collaborative software inspection tool . Experiences using the SuiteSound environment and applications are detailed , a retrospective of the difficulties and benefits of including multimedia in an object-based system is given , and suggestions for future work are included . 1 Introduction Computer Supported Collaborative Work -LRB- CSCW -RRB- is the study of methods for enhancing cooperation among computer users by providing tools that explicitly support interaction and the sharing of information -LSB- EGR91 , Taz88 -RSB- . Suite is a conventional object-based system with powerful tools for developing user interfaces that allow multiple users to manipulate distributed objects together -LSB- DV90 , DC91 -RSB- . This paper describes SuiteSound , a tool for manipulating multimedia flows , such as for digital audio , in Suite . Adding digital audio to Suite enhances distributed collaboration by helping participants work together more naturally . Since SuiteSound is integrated with Suite , applications can use sound in the way that is most natural for them . For instance , Collaborative Software Inspection -LRB- CSI -RRB- -LSB- DMRT91 -RSB- is a tool that supports distributed software inspections , using Suite for the software objects being inspected , and SuiteSound for audio -LRB- section 4.3 . -RRB- . CSI offers audio teleconferencing for participants to discuss the the documents during distributed inspection meeting , and audio annotations for participants to make comments on the documents while preparing for the meeting . There are many applications for multimedia flows in an object-oriented environment . Digital teleconferencing has already proven to be an effective way to hold meetings without traveling -LSB- Taz88 , Sak90 , AEH88 -RSB- . Multimedia electronic mail systems , such as Diamond -LSB- FT82 -RSB- , ease communication between participants , especially those who are not used to using computers . At Lawrence Livermore Laboratories , researchers are building a collaborative notebook for physical scientists . The scientists will be able to incorporate still images , live audio , and even live video in their notebooks , and readily share the notes with other researchers -LSB- FCB +91 -RSB- . Distributed education is also possible , with students at remote sites seeing video images of the instructor , and participating in the class through sound transmitted back to the classroom . For instance , collaborative visualization tools will allow the students to interactively explore algorithms with their instructors -LSB- Bro88 , TR91 , HR91 -RSB- . Digital audio will enable instructors to respond to questions verbally , rather than having to type replies . SuiteSound supports development of applications like these by integrating flexible , easyto-use digital audio with a flexible , easy-to-use object-based system . SuiteSound applications readily combine live or recorded audio with editable representations of user objects . This paper describes the design , implementation , and early experience with SuiteSound , along with some sample applications . The SuiteSound software is available for others who would like to experiment with multimedia flows in an object-based system . Purdue University gives away the Suite system along with SuiteSound1 . The University of Minnesota gives away SuiteSound to sites that already have Suite2 . 1.1 Previous work Many researchers have experimented with distributed digital audio . PhoneTalk is a multiuser system that runs on Sun Sparc Stations . A daemon runs on each machine and informs 1Prasun Dewan , Computer Sciences Department , Purdue University . pd@cs.purdue.edu . 2John Riedl , Computer Science Department , University of Minnesota . riedl@cs.umn.edu . receivers of the start of a session -LSB- NW90 -RSB- . A similar system called Vtalk supports voice talk between two Sun Sparc Stations . PhoneTalk and Vtalk have demonstrated the feasibility of teleconferencing tools that support voice interaction for multi-user tasks . The MERMAID system is designed for decision-making with multiple remote participants -LSB- Sak90 -RSB- . Data , voice , and video are transmitted using N-ISDN channels with 64Kpbs per channel . The voice and data use one B-channel and video the other . Users can talk freely , with voices mixed and transmitted to all users simultaneously . The Etherphone system supports locally distributed computing environments with multiple workstations and multiple networks . In the Etherpone system , each workstation is associated with an Etherphone that digitizes , packetizes , and encrypts the voice and transmits it over the Ethernet . The voice manager in the Etherphone system provides facilities for recording , editing , and playing stored voice -LSB- TS88 -RSB- . The Diamond multimedia message system supports creating , editing , transmitting , printing , and managing multimedia documents . A document may contain various media such as text , graphics , images , and speech -LSB- FT82 -RSB- . 1.2 Overview Our work extends previous research in four ways . First , SuiteSound provides an objectoriented interface for building applications that integrate multimedia with traditional objects . Second , our work supports multimedia flows with multiple sources and multiple sinks , and many different ways to merge or split the flows . Third , we are investigating methods of concurrency control that allow end-users to effectively control multiple multimedia streams at their workstations . Fourth , we are performing experiments to measure the costs of different implementation strategies for multimedia systems on conventional operating systems . Section 2 describes issues in incorporating multimedia in an object-based system on a conventional operating system . Many of these ideas are independent of the particular system we have chosen for our implementation . Section 3 discusses the design and implementation of SuiteSound . SuiteSound demonstrates several ways in which asynchronous behavior can be effectively incorporated in an object-based system . Section 4 describes our experience with three applications : Teleconf , Annotator , and CSI . We also talk about several simple pilot tests we have performed to understand the strengths and weaknesses of SuiteSound . Finally , section 5 summarizes our experience incorporating multimedia flows in a conventional operating system , discusses some of the lessons we have learned , and proposes future work . 2 Design Considerations 2.1 Continuous Media in Discrete Objects Object-based systems are being used for many tasks because of the benefits of improved encapsulation -LSB- Weg87 -RSB- . However , objects are inherently discrete , conceptually containing the representation of the state of a system at a fixed point in space and time . Continuous media , such as voice and video , flow across time , with no convenient boundaries for object representation . Objects can be used to represent discrete parts of the flow , but do not readily represent the entire flow . 4 SuiteSound breaks the flow of continuous media into chunks across time . Rather than using individual objects to represent these chunks , individual objects represent points in space past which the chunks must flow over time . Sound processing in each object is initiated by specifying a source and a callback method that will route the flow . From then on , the flow automatically moves through the object . The permanent state of the object never represents more than a brief interlude -LRB- usually 1/4 to 1 second -RRB- in the continuous flow of the media . Objects often keep summary statistics on the flow to help them in decisions such as floor control , but the flow is represented only as a series of source/destination decisions made in intermediate objects between the flow source and the flow sink . Source and sink objects can operate either asynchronously , performing callbacks to other objects at physical device speeds , or synchronously , transferring data to other objects at their request . Generally a flow has either an asynchronous source or an asynchronous sink driving the flow . If the source is asynchronous it pushes data through the flow towards the sink . If the sink is asynchronous it pulls data through the flow from the source . For instance , the teleconf object in floor control mode uses a real-time source , with sinks accepting data passively . Conversely , the annotator object when playing uses a real-time source to pull the data at the correct rate . There are several unusual flow types that do not fit this model . For instance , copying data between two annotator objects can be done with a non-real-time flow . Also , some applications have both real-time sources and sinks , and must flow-control between them -LRB- see section 4.1 -RRB- . SuiteSound provides a mechanism for creating flows , but does not restrict the user to a particular way of using them . Intermediate objects in the flow are referred to as filters . Filters process a flow entirely asynchronously , passing data through at the request of the objects on either side of them in the flow . Filter objects can manipulate the data either when they receive it or when they send it , and can also split the flow . Several types of splits are possible . For audio , the most common is a replication split , in which all of the data goes to multiple destinations . For instance , a replication split can be used to form a tee in which sound is recorded and simultaneously forwarded for a user to hear . Alternately , a frame split can reduce the sampling rate to reduce network load . 2.2 Floor Control Floor control in a collaborative system comprises a set of policies to control access to the set of shared objects . Floor control policies usually permit or deny access to all the objects in the collaboration simultaneously , conceptually controlling the entire \ floor '' at once . More general concurrency control policies separate the objects into domains that can be accessed individually -LSB- EG89 -RSB- . There are three reasons to use floor control for on-line sound : listener overload , bandwidth , and more effective conferencing . Listener overload refers to the difficulty in understanding multiple sound sources simultaneously . When visual cues are unavailable to help decide who should speak next , it is common for two speakers to begin at the same time . With no stereo separation to help locate the speakers , the listener has a difficult time interpreting the intermingled voice . Floor control can replace the visual cues with explicit requests to speak . Bandwidth is an issue because live sound runs at relatively high data rates . Having open channels between all pairs of participants is very expensive in computing and networking resources . Floor control can reduce the requirements from n2 open channels to n open channels . Finally , floor control can actively support more effective interaction . Systems such as Colab -LSB- SFB +87 -RSB- have demonstrated the benefits of explicit support for decision making systems . Floor control tactics can ensure that everyone gets a turn to speak , or can allow a meeting chair to directly control the meeting . There are several interesting types of floor control for on-line audio : FIFO - The floor is passed according to a queue of requests in a First-In-First-Out manner . The advantage of this mode is its fairness . Everybody gets an equal chance to hold the floor . Moderated - In this mode one participant has clear authority over the meeting and other participants and chooses the next floor-holder from an internal queue that is only visible to him/her . The moderator may wish to take the floor back at any time to give to another user . The floor always goes back to the moderator after a floor-holder is done speaking . Free - In this mode a number of participants may speak simultaneously and the result of their contention is unpredictable . We may have the voice come out garbled at the other end . FIFO with priorities - A variation of FIFO that allows for placement on the queue to be done according to a pre-determined scheme . For instance , the vice president would get a chance to speak before the student employee if both had a comment . Baton - The floor holder decides who gets the control next . The disadvantage of this mode is the subjective nature of this choice as two or more participants may monopolize the floor , leading to starvation -LRB- inability to voice their opinions -RRB- for some participants . 2.3 Speech Chunk Detection Many of the floor control modes can be simplified for users by automatically requesting the floor when speech is detected , and automatically releasing the floor when a silent period is detected . For instance , FIFO mode could be implemented with no user interaction , or Baton mode could automatically release the floor to the next holder after a few seconds of silence . Information about silent periods could also be used as feedback to the user . For instance , a floor-holder who has been silent for a few seconds could hear a beep on his workstation to remind him to release the floor . There are many possible algorithms for determining non-silent periods . Most are based on watching the linear audio signal for sections of pre-determined length with non-zero amplitude . We call non-silent periods speech chunks . We are investigating techniques based on exponential averaging of the linear signal , with different parameters for the averaging factor and the length of the non-zero signal required to indicate speech . We will report our initial experiences at the workshop . 3 Design and Use of SuiteSound The basic goal of SuiteSound is to provide flexible access to audio devices , within a distributed , object-oriented framework . Each participating workstation runs two SuiteSound objects , one for recording from the microphone and one for playing to the speaker . The objects provide simple facilities to initiate flows , as source or sink respectively . Once a flow is started , data can either be delivered automatically from a source to the next link in the flow , or pulled automatically by a sink from the previous link in the flow . Applications structure the flow as they desire . SuiteSound provides the mechanism for creating and managing flows , but does not enforce a particular structure on their use . Applications we have tested to date split flows , merge flows , store flows , and replay flows . Many other manipulations are possible , including sophisticated sound processing to improve quality or reduce network traffic . For instance , we are experimenting with a flow filter that will burst a flow into chunks of sequenced sound , leaving the silent periods out , to reduce network traffic . A sequencer filter at the sink will reconstruct the original flow so the timing is correct for the listener . The flow mechanism provides application designers with a higher-level view of the sound device , freeing them to build better applications that are integrated with the object-based environment . Sound flows are constructed as a series of objects linked together , with a source at one end and a sink at the other . In most applications either the source or the sink is chosen to provide the real-time clock rate for the flow . The designated clock source issues callbacks to the next link in the flow , delivering data if the clock source is also a sound source , and requesting data if the clock source is a sound sink . These method invocations from a sound object to an application filter are named callbacks because the method to be invoked is passed to another object , which \ calls-back '' the original object . Callbacks are similar to Swift 's upcalls -LSB- Cla85 -RSB- in that they are invocations from a lower layer of a protocol stack to a higher layer . Within an application , callbacks are used to pass the data from one filter to the next in the flow , until it arrives at the sink . Currently , all applications either use a physical audio device as a sink , in which case the sound is played at the end of the flow , or a recording filter , in which case the sound is usually made part of another flow at some later time . Future applications may use other sinks , such as an audio-to-text converter . 3.1 Suite - Distributed , Persistent Objects We used Suite to develop our application . Therefore , before describing SuiteSound , Teleconf , and Annotator , we will give a brief overview of Suite . Suite is a software system designed and developed at Purdue University by Prasun Dewan and his research team -LSB- Dew90 -RSB- . Suite supports remote procedure calls -LRB- RPC -RRB- , active persistent data , and management of \ direct manipulation '' user interfaces . A prototype of Suite has been implemented on top of UNIX , TCP/IP , NFS , and X -LSB- DV90 -RSB- . 3.1.1 Object Model The Suite object model is an extension of UNIX allowing distributed , shared , protected , and persistent objects . Suite objects coexist with existing components of the operating system , access conventional resources , and can be invoked by conventional processes -LSB- DC91 -RSB- . Suite has an object compiler that takes object programs , scans and interprets the specification of an object module in the object program , and generates an interface file and RPC stubs . Similarly , a dialogue manager compiler compiles the object program and generates object editors that allow interaction between the user and the object . 3.1.2 Components The components of Suite are : RPC - Suite RPC allows for applications executing in different address spaces and possibly on different hosts to name and communicate with each other by calling high-level remote procedures . Persistence - Suite objects are persistent in that their data structures can be checkpointed onto disk and later restored to memory . User Interface - The Suite user interface treats all objects as data that can be edited . Interaction with \ editable objects '' is made possible by dialogue managers -LRB- DMs -RRB- . DMs display a presentation of selected data structures , allow users to edit the presentation in a syntactically and semantically consistent fashion , and communicate these changes with the object . The object in turn ensures that the other displays also update their values . 3.2 Sound on the Sun SparcStation The Sun SparcStation provides a physical audio device that is capable of playing and recording sound at 8,000 octets3 per second . Each octet represents a single 13 bit Pulse Code Modulated -LRB- PCM -RRB- sample from the physical device , compressed to 8 bits using ? - law companding . SunOS 4.1.1 provides a library of functions for operating on the ? - law data , including functions for converting it to and from PCM so it can be manipulated linearly . For instance , in free floor mode SuiteSound 's teleconf utility converts to PCM to add simultaneous voice signals , and converts back to ? - law before playing on the device . The Sparcstation includes a built-in speaker , but a microphone is optional . Standard dynamic microphones and headphones work fine . Headsets are a substantial advantage because they keep the microphone near the speaker 's mouth , while leaving his or her hands free to use a mouse or keyboard4 . The interface to the audio device is through two special files called / dev/audio and / dev/audioctl . These files are opened and closed like other files . Once open , they can be read or written with the standard I/O calls . Reading from the device delivers encoded digital sound data from the microphone , while writing to the device delivers sound data to the speaker . Special control calls are available for checking the state of the audio device , 3An octet is an eight-bit byte . 4Plantronics , in Santa Cruz CA , is willing to make special headsets with an attached microphone that work with the Sun SparcStation typedef struct f unsigned num bytes ; unsigned char * byte opq ; g sound bytes type ; Figure 1 : This data structure is used for transmitting sound in Suite . changing the play or record gains , pausing during playing or recording5 , etc. . These control calls can be done on an opened audio device , or can be issued through the audioctl device . At most one process can have the audio device open for reading , and at most one process can have it open for writing . However , the audioctl device is sharable , so one application can issue control calls without interfering with other applications that are playing and recording sound data . The specifics of the audio device interface may change in future releases of SunOS . However , SuiteSound is compatible with any interface based on the general framework of a device to which blocks of sound can be read or written . The same SuiteSound interface can be implemented on many physical devices . 3.3 Recording Sound is transmitted in a flow in data structures known as sound bytes . Figure 1 shows the C declaration for this data structure . The Suite object compiler recognizes this struct as a declaration of a special data type called an opaque sequence . An opaque sequence is a variable-sized array , with a user-maintained count field . Opaque sequences are adapted from regular Suite sequences for data that does not require conversion on transmission . The count field is used in the RPC stubs to generate correct code to pack and unpack the sequence . Recorded data must be moved from the physical device to the first filter in the flow . The filter object that starts the flow identifies itself to the record object . Each time the record object has data available it delivers the data by invoking method UserRecordCallback in the filter object . There are two basic approaches to deciding when to deliver data . The simplest approach is to read data every time it is available on the sound device . This approach delivers sound in the lowest possible latency , but uses all of the computing resources of the workstation to do so . It is only appropriate if the sound application is the only application on the workstation . In the more common case that the sound application is supporting other applications , sound is delivered periodically in chunks of user-specified size . Since most communications costs are proportional to number of packets rather than number of bytes , there is a tradeoff between computing cost and latency of sound delivery . For most applications , the packets should be no smaller than 2000 bytes -LRB- about 14 second -RRB- and no larger than 8000 bytes -LRB- about 1 second -RRB- . The methods provided or used by the sound object for recording are : 5An undocumented feature is that close calls will hang if the audio device is paused . Async Method StartRecording : Begin delivering data from the sound device as fast as possible . Async Method StartSynchronousRecording : Begin delivering data from the sound device periodically , at each invocation of the background process . Async Method UserRecordCallback : Invoked in filter process to deliver sound data . Async Method StopRecording : Terminate recording . Async Method UserStopRecordCallback : Invoked in filter process after last sound packet has been delivered . UserStopRecordCallback is needed so the filter object can be sure the recording is completed . Otherwise , data that has already been recorded from the physical device when StopRecording is issued may be lost . 3.4 Playing The same sound bytes data structures are used for playing sound as for recording it . Again there are two options for transmitting data between the filter and sound objects . If the filter device is receiving the sound from a real-time source , it usually uses the synchronous mode of the sound device to play the data at the same rate it is being received from the source . Otherwise , the sound device is placed in asynchronous mode , so it will ask for at data at the rate that it can be played on the physical device . The methods provided and used by the sound device for playing are : Async Method StartPlaying : Start playing in asynchronous mode . Data is requested from the filter as needed . Async Method StopPlaying : Stop asynchronous play . Async Method UserPlayCallback : Invoked in the filter object to request data . Async Method PlayData : Data from the filter object , in response to an asynchronous callback . Async Method StartSynchronousPlaying : Start playing in synchronous mode . Data is played as it is delivered by the filter . Async Method StopSynchronousPlaying : Stop synchronous play . Async Method UserStopPlayCallback : Invoked in the filter object after the last data is played . Async Method SynchronousPlay : Play this data as soon as possible . 3.5 Sharing the Audio Device SuiteSound objects can be accessed transparently across the network . The sound is played or recorded on the workstation on which the SuiteSound object is physically located . To simplify access to correct SuiteSound object , they are named suite-sound - <operation> - <hostname> . For instance , a workstation named hannibal normally runs SuiteSound objects named suite-sound-play-hannibal and suite-sound-record-hannibal . SuiteSound currently only supports one active reader and one active writer at a time . SuiteSound opens the audio device for reading or writing when an object requests recording or playing . When one object stops recording or playing , another object can gain control of the SuiteSound object . This interface can be extended in several ways to improve access control : 1 . Users with transient applications -LCB- such as alarm clocks or mail \ biffers '' -LCB- would prefer that these short-term requests could over-ride long-term applications such as teleconf . 2 . Objects should be able to block to wait until a SuiteSound application releases the object . Objects that use SuiteSound could be assigned priorities , so the queue would better represent user 's desires . 3 . A simple filter application could allow compatible objects to share the device by overlaying the sound . For instance , background music could continue while an audio alarm clock plays its message . Such a filter is used within the teleconf tool to merge simultaneous speech . 4 Experience This section describes our experience using Teleconf , Annotator and CSI , and discusses some issues concerning the use of sound as a medium . 4.1 Teleconf Teleconf is an audio teleconferencing tool built on SuiteSound . Teleconf objects manage floor control for the users , and pass sound among themselves before playing it on the appropriate workstations . Figure 2 depicts the interconnections between SuiteSound objects during a teleconference . Each workstation runs two SuiteSound processes that manage the record and play capabilities of the device . In addition , each user has two teleconf objects , one for routing sound to other users , and one for receiving sound from other users6 . A master object manages floor control , not processing any sound itself , but informing the teleconf objects when they should transmit or receive sound . The teleconferencing tool currently supports two floor control modes : FIFO queuing and no floor control . In FIFO mode , participants press a button in a Suite dialog manager 6Section 5.1 discusses limitations in the UNIX process model that make it necessary to have two teleconf objects . floor requests grantedfloor grantedfloor floor requests ManagerDialogue ManagerDialogue play@alex SuiteSound record@alex SuiteSound play@alex Teleconf record@alex Teleconf master play@hanna Teleconf record@hanna Teleconf play@hanna SuiteSound record@hanna SuiteSound Request/Release floor , Exit conference Request/Release floor , Exit Conference recording sync/async playing recordiing playing playing sync/async playing Figure 2 : The SuiteSound teleconferencing tool . when they wish to speak . Then they join a queue of participants waiting to speak in the master object . The participant at the front of the queue holds the floor . The teleconf object for the floor-holding participant invokes the StartSynchronousRecording method on the participant 's SuiteSound object . Teleconf objects on all other workstations invoke the StartSynchronousPlaying method on their SuiteSound objects . The floor-holding teleconf object then begins receiving sound from UserRecordCallback methods invoked by its SuiteSound object . This sound is transmitted to each of the other teleconf objects , which pass it to their SuiteSound objects with SynchronousPlay method invocations . When the participant is finished speaking , he or she presses a button in a Suite dialog manager to release the floor . The teleconf objects all stop their SuiteSound objects7 . If there is another participant in the queue , the master assigns the floor again , and the process is repeated . The master is still involved in free mode , but now each member of the queue is granted the floor simultaneously . If there is any floor holder , all participant teleconf objects invoke StartPlayingon their SuiteSound object . All floor holders invoke StartSynchronousRecording on their SuiteSound objects , and begin transmitting sound to all other teleconf objects . Now , however , the teleconf objects have multiple incoming streams representing audio data that is being recorded simultaneously . They convert the data to linear form , and add it together with other data received since the last UserPlayCallback from the SuiteSound object . When the next UserPlayCallback arrives , the summed data is passed to the sound object to play . Figure 3 shows on all of the objects involved in the use of Teleconf . Not all of these dialogue managers are visible in a normal Teleconf session . The only window that has to be visible is the one teleconf window -LRB- either play or record -RRB- that supports the requesting and releasing the floor . The following are some of our observations regarding the use of the teleconferencing tool : Number of participants in FIFO mode - We have found out that when using FIFO , the waiting period for the individual participant to voice his/her comments becomes larger as the number of participants grows . The waiting can lead to a participant getting distracted not being able to follow the line of discussion , to a participant 's comments not being relevant to the present discussion anymore -LRB- as the line of discussion changes -RRB- or to a participant simply forgetting his or her comment . In our opinion , the maximum number of people who could hold a discussion effectively and comfortably is four . Restrictiveness of FIFO - FIFO is found to be too restrictive by our users , and in many instances takes the spontaneity away from the users . Delay - We held our first meeting in the same room and therefore were able to hear our own voice coming out of other machine a second later . This delay was noticeable by the participants including the speaker . However , when participants are distributed , they no longer hear their own voice and therefore can not detect a delay -LRB- as they do not know when the sound was transmitted -RRB- . User interface - The use of Teleconf should be transparent to the users . Because the tool is usually used to support some other activity -LRB- e.g. , software inspection -RRB- , the user 7In the original design , the floor was kept active if another participant was expected to gain the floor . The new design is simpler , and the slight increase in control messages does not significantly affect performance . Figure 3 : The Teleconf and SuiteSound objects in action will often be watching parts of the screen other than the Teleconf dialogue manager . Teleconf should mimic face-to-face speech as much as possible by providing audible feedback instead of textual feedback . Teleconf will support this notion in two ways : 1 . - Upon receiving the floor , Teleconf will inform the user by calling the user 's name at his or her workstation . 2 . - In FIFO mode , a floor holder who is inactive for more than two seconds -LRB- something that could happen if a person has forgotten to release the floor -RRB- , the user will hear a beep every two seconds . Merged sound - In free mode the speech of two speakers speaking simultaneously is merged . We intended that the speech would be garbled so the speakers would understand and resolve the problem , and so that listeners would know that someone else was waiting to speak . Surprisingly , simple speech was sometimes understandable when merged , and it was sometimes possible to identify the speakers in merged speech -LRB- an advantage for later floor control negotiation -RRB- . Free mode - Prior to making enhancements to read/write XDR functions , the performance of the tool in Free mode even with only two participants was intolerable . The words were chopped off at the end and the delay was more than noticeable . The enhancements resolved the problems mentioned above and made the tool usable . However , Free mode puts considerable load on the machines and as the number of participants grows to more than three , the machines will be unable to keep up . We concur with other researchers that teleconferencing with no floor control is difficult with more than three participants -LSB- Sak90 -RSB- . We are performing experiments to measure the load per user in free mode . We are also investigating extensions to free mode in which sound data representing relatively silent periods is not transmitted , reducing the load considerably . The following tables show a summary of the flat profile of all functions in Teleconf and SuiteSound objects in Free mode taking more than 1 % of the execution time . The information presented are the name of the functions , percentage of the running time of the program spent in the function , the number of times the function was invoked , and the average number of milliseconds spent in this function and its descendants per call . Originally , the sound bytes were declared to be of type character . This created performance problems as each byte was converted individually , resulting in several procedure calls and high time and space overheads . Extending Suite with the opaque data type , and declaring the sound bytes to be of type opaque has improved the performance . Now a single procedure call moves the entire block of sound data into the XDR data structures and the unnecessary conversions are avoided . There is a comparative table provided for the Teleconf object before the enhancements were done to the XDR functions . These tables do not tell the whole story . The original teleconf object in free mode with two participants took 25 % of the CPU time of a Sun Sparcstation . After the enhancement , the same teleconf object delivers the same sound data in 6 % of the CPU time . In both cases , most of the time is spent transmitting the data , but in the latter case the XDR overhead is gone . Function name Percent time calls total millisecond per call WriteBytes 71.1 2702 5.94 write 23.4 4836 0.86 select 1.8 2216 0.15 Table 1 : Profile of SuiteSound record object functions . Function name Percent time calls total millisecond per call XDR 35.5 398 47.50 Read32 16.3 Write32 15.8 mcount 14.9 read 11.7 4355 2.74 write 3.9 6345 0.63 select .6 1032 0.58 Table 2 : Profile of Teleconf functions before XDR enhancement . Function name Percent time calls total millisecond per call ReadBytes 32.3 9517 5.83 AddSoundBytes 23.8 1573 21.42 WriteBytes 21.2 7556 5.28 read 8.1 18770 0.61 write 8.0 21891 0.51 select 2.8 9393 0.43 Table 3 : Profile of Teleconf functions after XDR enhancement UserPlayCallback UserRecordCallback Synch Recording Async Playing ExitStart/Stop Playing Start/Stop Recording managerDialogue Play@alex SuiteSound record@alex SuiteSound Annotator Figure 4 : The annotator tool design . 4.2 Annotator The Annotator tool is a single-user application used for recording and playing back voice annotations . The voice entries are entered into a flexible size array in a sequential order and users can choose to play the recorded entries by selecting the index of any previously recorded message . The information stored and displayed for each annotation are the voice-entry-id , a key string that describes the annotation , and the time of creation . The annotator is implemented using the asynchronous play and synchronous record features of the SuiteSound object . In record mode the Annotator object calls the synchronous method StartSynchronousRecording in the SuiteSound object . The SuiteObject opens the audio device for reading and starts returning sound bytes to the caller object using the UserRecordCallback routine every half second . The received bytes are stored into the current entry of the SoundAnnotations array until there is no more room is available . For practical considerations , we decided on an annotation to be around ten seconds -LRB- i.e. , 80,000 bytes -RRB- with a maximum of ten voice entries allowed per session . The playing of information is done asynchronously . Once an index is chosen , the Annotator object invokes the asynchronous method StartPlaying in SuiteSound that checks the status of the audio device and prepares it for writing . The SuiteSound object calls UserPlayCallback every half second to receive sound at 5000 bytes chunks at a time . The sound chunk is sent back to the SuiteSound object for playing using the asynchronous method PlayData . This is done until there is no more data available on that entry , at which point we stop playing . Figure 4 depicts the interconnections between SuiteSound objects during an annotation session . As is the case with the Teleconf object , each workstation runs two SuiteSound processes that manage the record and play capabilities of the device . The user has one annotator object . A simple directory and lookup assistance will be provided with Annotator that finds a voice entry -LRB- i.e. , plays that entry -RRB- by looking up its identification number or the value of the key string . There will be two types of operations supported in Annotator : 1 . - Editing operations somehow change the present content of the sound annotations . Record - Recording of an entry . Concatenate - Concatenating of two existing entries . Replace - Replacing/overwriting an existing entry . Length - Returning the approximate length of an entry in milliseconds . Substring - Copying a substring of an existing entry into a new annotation . 2 . - The non-editing operations in no way change the content of the sound annotations , but may change the looks of things -LRB- i.e. , sorting the entries -RRB- . Play - Playing an existing entry . Loop - Looping/replaying over the same entry . Sort - Sorting the entries based on two keys : voice-entry-id or key string . Describe - Describing the content of a randomly chosen entry . The information given includes the key string , the length of time in milliseconds , and the time of creation . A heavy user of the Annotator will be the CSI tool . Annotator can provide asynchronous support for CSI by giving the reviewers a second way of recording their comments electronically -LRB- the first being textual -RRB- . The voice annotations are persistent and can be retrieved later during the course of the meeting . CSI can take advantage of the Annotator synchronously by storing all the voice interaction among the participants during the of Collaborative Software Inspection meeting for later play back . 4.3 CSI Collaborative Software Inspection -LRB- CSI -RRB- tool supports the process of software inspection in a distributed collborative environment . Traditionally , software inspection has been done with all the participants present in a meeting room and the inspection done on paper forms and documents . This approach is expensive in terms of traveling , scheduling , human resources , pre-meeting preparation of material , and post-meeting recording of the inspection results . We have designed and built a distributed collaborative software inspection , relaxing the requirement that all the participants be in the same room , providing automated support , and adding structure to the process . CSI allows geographically distributed participants to collaborate in the inspection and provides automated support for all inspection activities -LSB- DMRT91 -RSB- . Participants communicate through CSI dialog managers , teleconf , and voice annotation . Figure 5 depicts the screen of a participant in software inspection using the CSI and Teleconf tools . The asynchronous activities of software inspection include the distribution Figure 5 : The CSI and Telconf tools in action of target material and the creation of reviewer 's fault lists . The target material is on-line and accessible by reviewers who can use the browser to view it and annotate the lines of the target material appropriately . These annotations help the producer in the creation of the integrated fault list that will be used during the meeting . The synchronous activities of software inspection include discussion of the correlated faults , reaching a consensus on the faults , recording the action items , and finally determining the status of the inspection . The target material is brought on-line in a window on the screen of all the participants . Participants can comment on the target material either through a sound annotation or through typed annotations in a shared notepad . The recorder can also enter action items into an action list that is on-line and visible to all the participants . The recorder has a form that allows recording of information on the attendees , status of the meeting , and the final decision made concerning the target material being inspected . System activity -LRB- such as loading of files and objects , updating of data structures in objects , unloading of objects -RRB- is recorded automatically and is available for future statistical studies . We have several observations on the use of sound in CSI : Preferred mode - In software inspection participants play specific roles . One such role is that of a moderator . The moderator ensures that the meeting follows the established guidelines and does not stray from its line of objectives . Having a moderated floor control policy is best suited for software inspection as it allows the moderator to enforce certain protocols -LRB- i.e. , when a participants has held the floor for too long , the moderator may choose to take the floor away or simply be able to interrupt the speaker long enought to change the direction of the discussion -RRB- . Visual cues - We tried convening in the same room for the first time to make sure that things would go smoothly . We found out that people were accustomed to receiving visual cues and in effect would turn around to see one another while speaking . The later meetings were held -LRB- truly -RRB- distributed . We found out that when people do not see one another , they are more likely to focus on the actual work they are doing and consequently are more productive . Management - The management of the system is more difficult in a truly distributed environment where some of the users of the systems are less experienced than the others and are likely not to follow the established protocols or simply might press the wrong key . 5 Conclusion and Future Work 5.1 Problems and Advantages of Continuous Sound in Discrete Objects SuiteSound has proven a convenient tool for building object-based applications that include audio . The mechanism provided has made it relatively easy to build audio applications , and has proven flexible enough to support every application we have attempted . The flow model is convenient , especially since applications can initiate a flow and then continue with their other work while the flow passes through asynchronously . Suite has been a good platform for building collaborative applications , and the SuiteSound extension enhances these applications . However , we have found four limitations in our implementation , three introduced by limitations of the underlying UNIX process model , and one by the lack of dynamic binding in Suite . The problem with the UNIX process model is that it does not currently support multiple threads of control within an address space . This limitation means that methods within an object can interfere with the execution of other methods . Three examples of this problem are : 1 . Suite supports both synchronous and asynchronous method invocation . In synchronous method invocation , the invoking method waits for a reply from the invoked method before continuing . During this wait , no other method can be executed , thus blocking any flows that are in progress . Therefore , SuiteSound objects can not use synchronous methods . 2 . Real-time flows will sometimes run as fast as an object can process them , blocking other methods from executing . Thus , a user may not be able to stop the flow from a dialog manager . We used a feature of Suite by which the event dispatcher can be extended by an application to implement strict round-robin processing of method invocations . 3 . Flows with a cycle in them will sometimes fill up the TCP/IP streams used by Suite for communications , resulting in deadlock . To avoid this problem , our applications do not construct cyclic flows . For instance , the teleconf play and record objects are separate to avoid creating a false cycle . All of these problems would not exist in an operating system that supports multiple threads of control in each address space -LSB- CZ85 , Ras86 -RSB- . The fourth problem is that callback methods must be statically bound . For instance , when an object asks a SuiteSound record object to issue periodic callbacks , the name of the callback method is fixed at compile time . We work around this problem by using fixed names for callback procedures . An object tells its own name to the SuiteSound object , which then knows the callback method to invoke . Unfortunately , this makes it difficult to handle multiple simultaneous flows . This problem is difficult to solve , since the Suite object compiler checks type compatibility between methods at compile time . 5.2 Future work 5.2.1 Experiments We are interested in experimenting with audio flows to measure their cost and effectiveness on networks of different types . Interesting possible experiments include : Network requirements - Watch the actual network load while SuiteSound applications are running . Use a packet filter on a separate workstation attached to the same ethernet to measure the load caused by the sound packets . Computing requirements - Measure the computing costs of live audio on modern workstations . In practice , much of the cost of communication is computation -LSB- BMRS89 -RSB- . Since audio is most often used in support of other applications , it is important that it not put too much load on the system . Artificial delay - Introduce artificial delay and see if the participants are able to follow the conversation after such a situation . This experiment simulates the behavior of the system in an overloaded network , or across a wide-area network . User interface - Team up a novice user with a more experienced user of the tool and make a list of common mistakes and needs . This should help in understanding the type of user interface that is most suitable for the sound media . For instance , we have already discovered that it is best if most of the feedback provided by the system is audible output . Our experience is that usually sound is meant to support some other activity and the users are concentrating on the other activity . Service time - Measure the average \ time on the queue '' -LRB- from the time a user requests the floor to the time he/she gets serviced -RRB- . This experiment will be repeated with and compared against the various types of floor control and will help us in understanding what floor control policy is suitable for which type of a meeting . Floor control - Measure the effectiveness and satisfaction of participants solving problems under several different floor control policies , to understand the needs of the different types of meetings and how a particular floor control policy can best support a meeting type . Collect data on metrics such as service time , delay , and performance , and try to match a meeting type with its most suitable floor control policy . 5.2.2 Enhancements and extensions Teleconf , Annotator , and CSI are relatively new tools and as they are used and tested more , we will be enhancing and extending them further . Some of our work will include adding more forms of floor control to Teleconf and making its feedback to the user as audible as possible . Annotator will support different editing and non-editing operations including replace , sort , substring , describe , length , and concatenate . CSI will have more asynchronous support , including better use of the Annotator , to help the producer in preparing for the actual inspection meeting . The teleconf master is an interesting example of a collaborative tool in Suite , in that its implementation is completely independent of the teleconf application . We plan to incorporate the same master object into an existing text talk tool to provide similar floor control . This is an example of extending reusability in object-based systems to \ meta-objects '' that control the behavior of other objects . We are working with Prasun Dewan and his research group at Purdue University to develop concurrency control techniques that support the entire range of collaborative activity , using adaptability techniques to change concurrency control techniques dynamically -LSB- BR89 -RSB- . Part of that support is multimedia flows integrated with the objects in the system . We are actively exploring concurrency control techniques that are effective at presenting users with a consistent view of all the objects in the system , including multimedia flows . 5.2.3 Access control The issues of ownership and controlling access to protect those rights pose interesting problems when dealing with sound . Several important questions are : ffl Who may use a sound object on a workstation ? How can they be shared ? ffl Who can join a teleconference ? Which members should be listen-only ? ffl Who owns sound files generated during the course of a meeting , and if sound editing is allowed , who owns the resulting sound files . We are planning to work on these and many other problems and hope to report on our progress at the workshop . References -LSB- AEH88 -RSB- S.R. Ahuja , J. Ensor , and D. Horn . The Rapport multimedia conferencing system . In Proceedings of Conference on Office Information Systems , March 1988 . -LSB- BMRS89 -RSB- Bharat Bhargava , Enrique Mafla , John Riedl , and Bradley Sauder . Implementation and measurements of an efficient communication facility for distributed database systems . In Proceedings of the 5th IEEE Data Engineering Conference , Los Angeles , CA , February 1989 . -LSB- BR89 -RSB- Bharat Bhargava and John Riedl . A model for adaptable systems for transaction processing . IEEE Transactions on Knowledge and Data Engineering , December 1989 . -LSB- Bro88 -RSB- Marc Brown . Groupware . IEEE Computer , May 1988 . -LSB- Cla85 -RSB- D. Clark . The structuring of systems using upcalls . In Proceedings of the 10th ACM symposium on operating systems principles , December 1985 . -LSB- CZ85 -RSB- David . R. Cheriton and W. Zwaenepoel . Distributed process groups in the V kernel . ACM Transactions on Computer Systems , 3 -LRB- 2 -RRB- :77 -LCB- 107 , May 1985 . -LSB- DC91 -RSB- P. Dewan and R. Choudhary . Flexible user interface coupling in a collaborative system . Proceedings of the ACM CHI 's 91 Conference , April 1991 . -LSB- Dew90 -RSB- P. Dewan . A guide to Suite . Technical report , SERC-TR-60-P , Software Engineering Research Center at Purdue Univesity , February 1990 . -LSB- DMRT91 -RSB- J. Drake , V. Mashayekhi , J. Riedl , and W. Tsai . A Distributed Collaborative Software Inspection Tool : Design , Prototype , and Early Trial . Technical report , TR-91-30 , University of Minnesota , August 1991 . -LSB- DV90 -RSB- P. Dewan and E. Vasilik . An object model for conventional operating systems . Usenix Computin Systems , December 1990 . -LSB- EG89 -RSB- C.A. Ellis and S.J. Gibbs . Concurrency control in groupware systems . In Proceedings fo the ACM SIGMOD '89 Conference in Groupware systems , May 1989 . -LSB- EGR91 -RSB- C. Ellis , S. Gibbs , and G. Rein . Groupware . Communications of the ACM , January 1991 . -LSB- FCB +91 -RSB- J.W. Fickett , M.J. Cinkosky , C. Burks , P.E. Hempfner , D. Nelson , R.M. Pecherer , P. Sgro , D.M.Sorenson , R.D. Sutherland , and B.C. Yantis . Human Gnome Information Resource . Technical report , Obtained through personal communication , 1991 . -LSB- FT82 -RSB- H.C. Forsdick and R.H. Thomas . The design of Diamond : A distributed multimedia document system . Technical report , TR number 5402 , Bolt Beranek and Newman Inc. , October 1982 . -LSB- HR91 -RSB- J.C. Hamlin and John Riedl . Visualize : Computer visualization of graph theory , March 1991 . Undergraduate Research Opportunities Program , University of Minnesota . -LSB- NW90 -RSB- P. Nises and J. Wettby . Phonetalk . Technical report , Royal Institue of Technology , December 1990 . -LSB- Ras86 -RSB- Richard F. Rashid . Threads of a new system . Unix Review , 4 -LRB- 8 -RRB- :37 -LCB- 49 , August 1986 . -LSB- Sak90 -RSB- S. Sakata . Development and evaluation of an In-house multimedia desktop coneference . IEEE journal on selected areas in communcations , April 1990 . -LSB- SFB +87 -RSB- Mark Stefik , Gregg Foster , Daniel G. Bobrow , Kenneth Kahn , Stan Lanning , and Lucy Suchman . Beyond the chalkboard : Computer support for collaboration and problem solving in meetings . Communications of the ACM , 30 -LRB- 1 -RRB- :32 -LCB- 47 , January 1987 . -LSB- Taz88 -RSB- J.M. Tazelaar . In depth groupware . Byte magazine , December 1988 . -LSB- TR91 -RSB- Jamie Thingelstaad and John Riedl . Collaborative physics instruction on the macintosh , March 1991 . Undergraduate Research Opportunities Program , University of Minnesota . -LSB- TS88 -RSB- D.B. Terry and D.C. Swinehart . Managing stored voice in the Etherphone system . ACM Transactions on Computer Systems , February 1988 . -LSB- Weg87 -RSB- Wegner . Dimensions of object-based language design . In OOPSLA Proceedings , October 1987 .

B I O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B I O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B I O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

continuous media
digital audio
multimedia