From: bobp@hal.com (Bob Pendleton)
Subject: Re: TECH: Sound API?
Originator: hlab@stein.u.washington.edu
Organization: HaL Computer Systems, Inc.
Date: Fri, 9 Oct 1992 20:54:19 GMT



From article <1992Oct7.064606.28107@u.washington.edu>, by
thinman@netcom.com (Technically Sweet):

> There is no possibility of a universal low-level API for the various
> types of sound hardware available.  

I seriously doubt this. It might be very ugly, but it is doable. But,
that isn't what I'm trying to look at. I want to explore features of a
high level VR sound API. I think that looking at sound exposes a
number of problems that the graphics heavy point of view that
dominates these discussions overlooks.

> Various kinds are quite unique.
> You can't talk to a sampler the way you talk to an FM synthesizer.
> Check out a pro-level MIDI sequencer package.  It's really amazing the
> amount of custom info they have for each supported synth.

Yes, it is amazing. Have you ever looked at the special features of
say an E&S high graphics rendering system versus a Fujitsu high end
system, versus an Kubota system, versus a Shographics system? They are
all very different but they provide APIs that allow PHIGS and X based
programs to run. Yes, if you want to actually write code that runs on
all of them you have to either program using a common subset of the
APIs, or you have to do lot's of queries to find out what is
available...

But the queries are at least standard :-) and there is a standard way
for each company to register their extensions.

> An object which has sounds as part of it will have to query the deck
> and find out what kind of sound hardware is available, and if it knows
> how to make sound X on that hardware, great.

Of course. Can we come up with a list of valid query functions?  These
would need to be part of the API.

> I haven't considered the issues you raised but have concentrated on
> nitty-gritty aspects.  There are really two target deck configurations
> here: immersive & non-im.  The former needs a 3D audio system
> integrated with head-tracking; the former gets by fine with stereo
> speakers on the sides of the monitor.  Head-tracking is not needed
> here; it is in fact a bad idea.  So, straight stereo placement
> algorithms are fine.

Right, so the voice positioning part of the API has to be able to
query the deck to find out whether it is an immersive environment or
not and configure its self appropriately. Have you every looked at the
amount of information that is exchanged between an X client and an X
server during the connection dance?

> There are three simple ones available: volume panning, phase-based
> placement, and the Doppler effect.  All three are very easy
> computationally.  Formulae for these, and more complex elevation-based
> methods, are in "Elements of Computer Music" by F.  Richard Moore,
> Chapter 15, "Spatial somethingorother".  The current crop of stereo
> CD-quality sound cards for the PC are capable of these effects.  The
> Gravis Ultrasound is probably the one to pick (wait until it goes
> through initial product shakedown) as it allows you to load up 32
> sound samples and plays them for you.

Thanks for the reference and the product recommendation. 

> It was in thinking about sound I/O, texture-mapping, and real-time
> video, that I realized the need for a new metaphor for real-time
> software.  The traditional real-time OS provides a clockworks
> metaphor.  

I guess we come from different traditions. :-)

> We need a system based on hydraulics or chemical
> processing.  That's why I'm using a dataflow-based system.

I worked for a chemical company for a, brief, while. If you look at
"real" chemical plants you do find fluid flows. But you find the flows
are controlled by combinations of simple feedback system used to
control rates and by finite state machines used to respond to events.

You also find a lot of batch processes. Many chemical processes don't
work to well as continous processes.

Real time video and sound are analogous to fluid flows. The work I've
done in controlling their routing and display used finite state
machines that responded to events to control the sources, sinks, and
routings of the video and sound.

If you digitize the video and sound you've converted them from
continous flows to something that can be batch processed. Just so long
as you keep the batch rate high enough :-) Then you can have
discontinous flows that can be moved along in batch so long as you can
synchronize the batches at the delivery point.

I don't see how texture mapping fits into the dataflow model. I do
understand how rendering from a structure can be viewed as a dataflow
problem. But, to me texture mapping is just one step in the rendering
pipeline. (The fact that it's called a "pipeline" gives you a pretty
good idea that rendering is viewed as a "flow" type problem.)

Come on, never say never.

                                Bob P.
-- 
Bob Pendleton             | As an engineer I hate to hear:
bobp@hal.com              |   1) You've earned an "I told you so."
Speaking only for myself. |   2) Our customers don't do that.
                   <<< Odin, after the well of Mimir. >>>
Newsgroups: sci.virtual-worlds