EDIT: I didn't mean to sound so overconfident or showing off my credentials. This is my personal view given what I know, but there are things I don't know (like good vs. bad OS audio APIs, how to tune kernels and pick good audio drivers, and whether you can get away with allocations, mutexes, or a JITted/GC'd language on the audio thread). It's possible that writing audio code in JS or non-real-time C++ won't cause problems to users in practice, as long as it runs on a dedicated thread and doesn't contend with the GUI for the event loop, and processes notes synchronously and deterministically, but I haven't tried. For various opinions on this from people who have done more experimentation than me, see https://news.ycombinator.com/item?id=27128087 and http://www.rossbencina.com/code/real-time-audio-programming-....

----

I've spent many years working on computer music and DSP in general, and the last two years writing interactive audio apps like DAWs and trackers (traditionally written in real-time code).

Personally I feel that not only should the signal processing be handled in compiled code, but the process of allocating voices for incoming notes and sweeping filter parameters for existing notes (like a VST/VSTi plugin) should be real-time/allocation-free and deterministic.

Additionally for a sequencer or DAW-like program playing a scripted project file, the process of scanning through a document and determining the timing of events should also be real-time and deterministic, requiring it be synchronous with the code driving synthesis/notes/envelopes. (Some Web Audio music demos I've seen on HN, like the 64-step drum sequencer[1] or The Endless Acid Banger[2], are not deterministic. On a slow computer, when the browser's JS engine stutters, notes stop being triggered even though existing notes sustain.) I think that requires that the "document" is stored in C/C++/Rust structs and containers, rather than GC'd objects like in JS.

(Processing user input depends on low latency without latency spikes rather than determinism and synchronous processing, but I don't know if browsers are good at that either.)

At this point, I find it significantly easier to write a GUI in native toolkits like Qt, than to learn and write bindings for a GC'd language to access native data structures. And unfortunately there is a limited selection of mature native toolkits that are both not buggy (Qt is buggy) and has accessibility and internationalization, and optionally theming and native widgets. I still believe that writing a GUI in a native language can become a better user and developer experience than browsers, if more people invest effort and resources into better paradigms or implementations to fix native's weaknesses compared to browsers (buggy libraries, no billion-dollar megacorps funding web browsers, apparently people nowadays think that React using a virtual DOM and recomputing parts of the UI that don't change is a strength), while maintaining its strengths (less resources required, more predictable memory and CPU usage, and trivial binding to native code).

What's the current state of native desktop GUIs? Qt is nearly good enough, but is run by a company trying to antagonize the open-source community and rip off commercial users, binds you to C++ (which is less pleasant than Rust), suffers from bugs (some neglected, deep-seated API design flaws), and handles reactivity poorly (though QProperty and transactions[3] promise to improve that aspect). GTK has cross-language support, but in gtk-rs, GTK's refcounted design and inheritance and Rust's move-based design and composition are fighting against each other. There's other older APIs like WxWidgets, FLTK, FOX, etc, many of which I personally dislike as a user. Flutter is promising, but still buggy, the upper layers are virtual-DOM-based (I have reservations), and feels foreign on desktops (many missing keyboard shortcuts, one app (FluffyChat's unfinished desktop port) is missing right-click menus, has broken keybinds, and has painfully slow animations).

Is there an alternative? Where would you draw the seam between a GC'd GUI and a realtime audio engine, to minimize the boilerplate glue code, and ensure that note processing and audio synthesis are real-time and deterministic (does this require being synchronous with each other?)?

I've seen a lot of software with a non-real-time or nondeterministic audio engine/sequencer (even though I personally think it's bad for users and unacceptable when I design a program). For example, ZorroTracker has an audio processing path written in JS (a RtAudio thread calling into V8 running in parallel with the GUI thread, calling JS bindings to native .dll/.so audio generators), coupled with a (planned) sequencer written in JS and operating on JS objects, giving up on real-time. As I've mentioned, several browser-based audio projects I've seen on HN generate audio in Web Audio (real-time), but trigger notes asynchronously in non-real-time JS code.

> Eventually it dawned on me that the task of composing audio processing blocks is itself free of those realtime performance constraints mentioned above: it's only the actual rendering step that must meet such requirements.

Given my current understanding of real-time programming, I think that the task of feeding inputs into audio processing blocks is not free of realtime performance constraints. In any program with an audio sequencer, I'm not aware of how to get deterministic playback with predictable CPU runtime (which I do not think is worth compromising), without worrying about "memory management, object lifetimes, thread safety, etc.", preallocating memory for synthesis, and picking a way to communicate with the UI (eg. lock-free queues for processing commands sequentially, and atomics or triple buffers for viewing the latest state). If you have a solution, do let me know!

[1]: https://news.ycombinator.com/item?id=27112573

[2]: https://news.ycombinator.com/item?id=26870666

[3]: https://doc-snapshots.qt.io/qt6-dev/template-typename-t-qpro...