At 48 kHz, Web Audio gives an AudioWorklet 2.667 ms to produce 128 samples. There is no retry. If process() has not returned when the device needs the next block, the listener hears a click.

nomus runs a recurrent network with 1,037,475 parameters inside that deadline, in a Chrome extension and on a public demo page. It removes background music from a tab and keeps the voices. Measured in Node/V8 through the same WebAssembly interface the worklet uses, a stereo 128-sample block costs about 48 µs on average, with a 99th percentile of 190 to 210 µs, against a 2,667 µs budget. The engine adds exactly 20 ms of delay, in every mode.

This post is about how that fits: a Rust engine compiled to a WebAssembly module with no imports, a flat C ABI instead of generated bindings, and a worklet written on the assumption that anything can fail.

The budget, and what it rules out

A render quantum is 128 samples. At 48 kHz that is 128 / 48,000 s = 2,667 µs. Meeting it on average is not enough; it has to be met every time. That rules out anything whose worst case is unbounded:

  • No allocation on the hot path. malloc can take a lock or ask the kernel for pages.
  • No locks. A worklet waiting on a lock held by the UI thread inherits the UI thread’s scheduling.
  • No garbage. A JavaScript allocation per block is work for the collector, on the thread where a pause is audible.

The Rust engine allocates every buffer when a processor is built. Per block it copies samples between preallocated arrays and runs the network. There is no Mutex anywhere in the core crate.

Where the 20 ms comes from

The engine works in frames of 480 samples (10 ms). Its analysis window is 960 samples, two frames, so a hop of output is ready one hop after its input arrives. That costs one frame of algorithmic delay. One more frame is buffering: the output FIFO starts primed with 480 zeros, so every sample waits the same time whatever the block size.

480 + 480 = 960 samples, 20 ms. Every mode is padded to the same total, including bypass. That is deliberate. With a constant delay, switching between the filter and the original is only a question of which timeline you listen to, and a switch never repeats or skips audio.

480 is not a multiple of 128. Fifteen quanta carry exactly four frames:

Quantum 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
Frame runs yes yes yes yes

So the cost is spiky. Eleven of fifteen quanta only copy samples; four run the whole network for every channel. A frame has to finish inside the quantum that completes it, which is why the numbers to watch are the p99 and the maximum, not the mean. Natively on an Apple M4 Pro, the voice mask averages 37.7 µs per 128-sample stereo block, which is about 141 µs per stereo frame.

“20 ms end to end” is the engine’s share. The extension’s popup reports the total as the engine’s 20 ms plus the AudioContext’s baseLatency and outputLatency. Wireless headphones add delay the browser cannot see.

A module that imports nothing

Most Rust-to-browser projects use wasm-bindgen. nomus does not, for two reasons the crate documentation states. The AudioWorkletGlobalScope has no fetch and no TextDecoder, which the generated glue expects. And the engine never needs a string or an object to cross the boundary. Its whole interface is “here are 128 floats, give me 128 back”, plus a few integers.

The build target is wasm32-unknown-unknown, Rust with no operating system. Allocation and maths work; threads, files and clocks do not. The core crate’s two dependencies, realfft and nnnoiseless, are plain computation, so the module needs nothing from its host. You can check that in one line:

node -e 'const m = new WebAssembly.Module(require("fs").readFileSync("extension/pkg/nomus_wasm.wasm")); console.log(WebAssembly.Module.imports(m), WebAssembly.Module.exports(m).length)'

It prints [] 20: no imports, and 20 exports (the memory and 19 functions). The module is 815,166 bytes. Zero imports is a property you keep by checking it after every dependency change. If an import from env or __wbindgen_placeholder__ ever appears, instantiating with an empty import object fails with a LinkError.

One browser detail: an extension page always runs under a content security policy, and compiling WebAssembly from bytes counts as a form of eval. The manifest’s policy lists 'wasm-unsafe-eval', which allows WebAssembly compilation and nothing else.

A flat C ABI

Every export takes and returns integers or f32. An engine is a handle: nomus_create boxes an engine inside linear memory and returns its address, which JavaScript stores as a number. Sample buffers are allocated by the host through nomus_alloc, once, during initialisation, so the audio path never allocates a buffer.

The first export is a version. A hand-written ABI has no compiler checking both sides, so the worklet compares nomus_abi_version() with its own constant before calling anything else, and refuses to run on a mismatch.

The hot path is one function, called once per channel per quantum:

#[no_mangle]
pub unsafe extern "C" fn nomus_process(
    engine: *mut Engine,
    channel: u32,
    input: *const f32,
    output: *mut f32,
    n: u32,
) -> u32 {
    let Some(e) = engine.as_mut() else {
        return 0;
    };
    if input.is_null() || output.is_null() {
        return 0;
    }
    let n = n as usize;
    if n > Engine::MAX_BLOCK || channel as usize >= e.channel_count() {
        return 0;
    }
    let bytes = n * std::mem::size_of::<f32>();
    let overlaps = (input as usize).abs_diff(output as usize) < bytes;
    if overlaps {
        // Finish reading before constructing any mutable reference, including
        // for partially overlapping buffers. Keep allocation off the audio path.
        let mut tmp = [0.0; Engine::MAX_BLOCK];
        std::ptr::copy_nonoverlapping(input, tmp.as_mut_ptr(), n);
        let out = std::slice::from_raw_parts_mut(output, n);
        return e.process_channel(channel as usize, &tmp[..n], out) as u32;
    }
    let inp = std::slice::from_raw_parts(input, n);
    let out = std::slice::from_raw_parts_mut(output, n);
    e.process_channel(channel as usize, inp, out) as u32
}

It validates before touching memory. It also handles overlapping buffers. In Rust, holding a shared and a mutable slice over the same bytes is undefined behaviour, even if every sample is read before it is written. So an overlapping input is first copied to a 16 KiB stack array. The worklet always passes separate buffers; the aliasing path exists so the ABI is sound for any caller. Two native tests cover it: exact and partial overlaps must produce the same output as separate buffers, and an oversized block must be rejected before any buffer is read.

Getting the engine onto the audio thread

The worklet cannot fetch its own module. The main thread fetches and compiles it, then hands the compiled WebAssembly.Module to the worklet in processorOptions, along with the model bytes. The worklet instantiates it synchronously with an empty import object.

We tried the other route first. Posting a WebAssembly.Module through the node’s MessagePort is silently dropped by some Chromium builds: no error on either side, the message never arrives. Two rules came out of that:

  • Make readiness explicit. The worklet posts ready only after the engine exists, and every host waits for it with a timeout: 5 seconds in the extension, 8 on the site.
  • Audio that plays is not proof the engine runs. Before it is ready, the worklet passes input through. On a clip without music, that sounds exactly like a working filter. Trust the ready message and the frame counter, not your ears.

The worklet also checks the sample rate. The model learned what voice looks like in 481 bins spaced 50 Hz apart at 48 kHz. Every host creates its context with sampleRate: 48000, so Chrome resamples the tab into it. If a host forgets, the worklet posts a warning.

Memory that moves under your views

JavaScript reads and writes the engine’s buffers through Float32Array views over memory.buffer. When the allocator inside the module grows memory, the browser replaces that ArrayBuffer and detaches the old one. Every view over it becomes empty, and writing to one throws.

Growth is not rare. Running the shipped module in Node and printing the page count (64 KiB each) after each step:

Step Pages
Instantiate 18
nomus_create(2) 22
nomus_process on the fourth block, when the first frame runs 23
Allocate 4,150,261 bytes for a model 90
nomus_load_model 218

The third row is the one that matters: memory grew inside nomus_process, on the audio path, the first time a frame ran. A view created before that call was dead after it. The fix is an identity check, run at the start of each quantum and after every call:

  // WebAssembly.Memory may grow (detaching old views); re-create on change.
  refreshViews() {
    const buf = this.exports.memory.buffer;
    if (buf !== this.memBuf) {
      this.memBuf = buf;
      this.inView = new Float32Array(buf, this.inPtr, BLOCK);
      this.outView = new Float32Array(buf, this.outPtr, BLOCK);
    }
  }

Creating fresh views every block would also work, and would allocate 1,500 small objects per second in stereo on the audio thread. The comparison allocates nothing in steady state.

Never throw on the audio thread

If process() throws, the browser never calls that processor again. The node stays connected and silent. In the extension that means a muted tab, because while Chrome captures a tab it plays that tab’s audio only through the capture. So the real work is wrapped:

  process(inputs, outputs) {
    this.processCalls++;
    try {
      return this.render(inputs, outputs);
    } catch (err) {
      const msg = String(err && err.message ? err.message : err);
      if (this.lastError !== msg) {
        this.lastError = msg;
        this.port.postMessage({ type: "error", error: `process(): ${msg}` });
      }
      const input = inputs[0];
      const output = outputs[0];
      if (input && input.length && output) {
        for (let c = 0; c < output.length; c++) output[c].set(input[Math.min(c, input.length - 1)]);
      }
      return true;
    }
  }

On any exception the listener hears unfiltered audio, the error is reported once per distinct message, and the processor stays alive. A Rust panic compiles to a WebAssembly trap under panic = "abort" and arrives here as a RuntimeError.

Measuring without a clock

The worklet scope may not have performance.now(). The worklet feature-detects it and falls back to Date.now(). With a millisecond clock and a 2.667 ms quantum, nearly every measured block would read 0 ms and an occasional one 1 ms. So when the timer is coarse, the worklet reports its load as null instead of a number that looks precise and is not. The popup measures cost on the main thread instead, on a second instance of the same compiled module, which shares no memory with the one on the audio thread.

The extension around it

A Manifest V3 extension splits this across four contexts, each doing what only it can:

The popup holds the user’s click, which Chrome requires before it hands out a capture stream. Only the service worker may create the offscreen document, and only the offscreen document can hold an AudioContext that outlives the popup.

The start function in the offscreen document is a chain of awaits, and a stop or a newer start can overtake any of them. Each attempt gets a generation number, and after every await it checks that it is still current, so a stream granted after the user clicked Stop is cleaned up instead of played. The graph is wired only after ready. Four tests run this file unchanged in Node against fake browser APIs, including a stop that arrives while getUserMedia is still pending.

Privacy is enforced rather than promised. The policy’s connect-src 'self' blocks network connections to any other origin, the module has no imports it could call out through, and the worklet scope has no fetch. The extension has no host permissions and no content scripts.

Proving the browser build computes the same thing

Parity between Rust and PyTorch on the native build does not prove the .wasm matches: a different target, SIMD code generation and the ABI sit in between. A check script drives the real module through the real ABI in Node. It loads the model exactly as the worklet does, asserts the 960-sample latency, streams a test vector through block sizes of 1, 128, 511 and 1,024 samples with a reset between them, and compares the output with the waveform PyTorch computed. The worst error must be below 1e-5. For the current model it is 8.6e-7.

The same script times 3,000 warmed stereo blocks. Its own output says what those numbers are not: Node/V8, not live Chrome. V8 compiles WebAssembly the same way in Chrome, but the audio thread’s scheduling, other tabs and power state are not part of the measurement.

Rough edges

  • An unused name section. The build does not strip symbol names, so 98,500 bytes of function names, about 12% of the module, ship with it. strip = true in the release profile removes them.
  • Mode switches allocate. Switching modes builds a new processor chain and crossfades to it. The worklet’s message handler runs on the audio thread, so that allocation happens there, once per switch, never per block.
  • Strength is not smoothed. It is read once per 10 ms frame, so a fast slider move changes gain in steps.
  • The worklet has no unit tests. The ABI under it is tested natively and through the check script; the processor itself is exercised through a local test page, the site demo and the extension.

Try it

The nomus demo runs this module and this worklet in your browser on sample clips, with the same 48 kHz context, the same ready handshake and the same model. Nothing is uploaded. The extension is on the nomus page; the desktop and Android apps are coming soon.

If you are putting real-time work in a browser, the parts that transfer are small: keep the module free of imports and check it after every dependency change, allocate at set-up and never in the callback, re-check your memory views after any call that can allocate, and never let the audio callback throw.

Agrohi builds production systems like this for startups and teams. Tell us what you are building.