Build a Gamepad Instrument With Web Audio (No MIDI Required)

Every browser gamepad tutorial routes through MIDI, and most of the time that is the wrong architecture. If the sound is going to come out of the same tab the controller is read in, you do not need MIDI at all — you need the Gamepad API and Web Audio, and nothing else. No virtual port. No permission prompt. No Chromium requirement. It works in Safari. It works on an iPad. This post builds a genuinely playable instrument that way, and is honest about where the browser's audio clock stops being your friend.

Why skip MIDI entirely?

Because MIDI is a transport, and if the producer and the consumer are in the same JavaScript context there is nothing to transport.

Routing a gamepad through Web MIDI to make sound in the same page means serialising a number into three bytes, handing it to the OS MIDI subsystem, having it delivered back to you, and parsing it again. You get a looser clock, a permission prompt, a virtual port dependency, and a Chromium-or-Firefox-desktop requirement — in exchange for nothing, because you could have called a function.

MIDI earns its place the moment something else needs to hear the messages: a DAW, a hardware synth, another application. That is the case the browser bridge guide covers. If the sound is staying in the tab, cut it out.

The reach difference is substantial too. The Gamepad API sits around 97% globally including Safari and iOS, and Web Audio is similarly broad. Web MIDI is around 79% and excludes Safari entirely. A gamepad instrument is the one browser music rig that works on an iPad.

How do you set up the AudioContext correctly?

Construct it with the interactive latency hint, and resume it from a real user gesture before you expect any sound.

The latencyHint option accepts "interactive", "balanced", "playback" or a number of seconds. MDN documents that it is a request the browser may ignore, and that you can read the effect back: "interactive" typically reports a very small baseLatency where "playback" reports a much larger one. For an instrument you always want "interactive", which is also the default.

The autoplay rule is the part that silently defeats people. Autoplay policy means a newly constructed context starts suspended. And a gamepad button press does not count as a user activation for this purpose — the browser's activation model is about DOM input events, and gamepad state is polled, not evented. So you need one genuine click or keypress to unlock audio, after which the controller takes over.

// audio/context.js
let ctx = null;

export function getContext() {
  ctx ??= new AudioContext({ latencyHint: 'interactive' });
  return ctx;
}

export async function unlock() {
  const c = getContext();
  if (c.state === 'suspended') await c.resume();
  return c.state === 'running';
}

export function latencyReport() {
  const c = getContext();
  return {
    sampleRate: c.sampleRate,
    // Seconds spent handing a buffer from the graph to the audio subsystem.
    baseLatency: c.baseLatency ?? null,
    // Estimated seconds from that handover to the first sample hitting the DAC.
    outputLatency: c.outputLatency ?? null
  };
}

// One real gesture unlocks everything. A gamepad press is not a gesture.
document.getElementById('start').addEventListener('click', async () => {
  const ok = await unlock();
  document.getElementById('status').textContent = ok
    ? 'Audio running. Press a controller button.'
    : 'Audio could not start.';
});

latencyReport() is worth keeping in your UI during development. These are measured values from the running context, not estimates from a blog post, and they tell you immediately whether the browser honoured your hint. outputLatency in particular is the number that changes when somebody switches from built-in speakers to Bluetooth headphones, and it is the honest answer to "why does it feel laggy on this machine".

What does one voice look like?

An oscillator into a gain into a filter into the destination, built fresh for each note and discarded when it finishes.

The critical constraint, and the one that trips up anyone coming from a traditional audio API: an OscillatorNode can be started exactly once and cannot be restarted after it stops. This is not a limitation to work around — it is the intended design. Nodes are cheap, the graph is garbage collected, and building per note is the idiomatic pattern.

// audio/voice.js
import { getContext } from './context.js';

const midiToHz = (n) => 440 * Math.pow(2, (n - 69) / 12);

export function createVoice({ note, velocity = 1, wave = 'sawtooth' }) {
  const ctx = getContext();
  const t = ctx.currentTime;

  const osc = ctx.createOscillator();
  const filter = ctx.createBiquadFilter();
  const amp = ctx.createGain();

  osc.type = wave;
  osc.frequency.setValueAtTime(midiToHz(note), t);

  filter.type = 'lowpass';
  filter.frequency.setValueAtTime(1200, t);
  filter.Q.setValueAtTime(4, t);

  // Attack / decay / sustain, all scheduled on the audio clock.
  const peak = 0.28 * velocity;
  amp.gain.setValueAtTime(0.0001, t);
  amp.gain.exponentialRampToValueAtTime(peak, t + 0.008);        // 8 ms attack
  amp.gain.exponentialRampToValueAtTime(peak * 0.6, t + 0.18);   // decay to sustain

  osc.connect(filter).connect(amp).connect(ctx.destination);
  osc.start(t);

  return {
    note,
    filter,
    osc,
    amp,
    release(when = ctx.currentTime, seconds = 0.25) {
      amp.gain.cancelScheduledValues(when);
      amp.gain.setValueAtTime(Math.max(0.0001, amp.gain.value), when);
      amp.gain.exponentialRampToValueAtTime(0.0001, when + seconds);
      osc.stop(when + seconds + 0.02);
    }
  };
}

Two details that separate a usable envelope from an amateur one. Everything is scheduled against ctx.currentTime, which is the audio clock, not performance.now() or Date.now() — those are main-thread clocks and have nothing to do with where the audio render is. And the gain ramps are exponential rather than linear, because perceived loudness is roughly logarithmic and a linear fade sounds like it falls off a cliff at the end.

The 0.0001 floor is not superstition either: exponentialRampToValueAtTime cannot ramp to or from zero, so you ramp to an inaudible epsilon instead.

How do you allocate voices from buttons?

Keep a map from button index to live voice, create on press, release on release, and cap the total.

// audio/polyphony.js
import { createVoice } from './voice.js';
import { getContext } from './context.js';

const MAX_VOICES = 8;
const voices = new Map();          // key -> voice

export function noteOn(key, note, velocity = 1) {
  if (voices.has(key)) noteOff(key);
  if (voices.size >= MAX_VOICES) {
    // Steal the oldest voice. Map preserves insertion order.
    const oldest = voices.keys().next().value;
    noteOff(oldest);
  }
  voices.set(key, createVoice({ note, velocity }));
}

export function noteOff(key) {
  const v = voices.get(key);
  if (!v) return;
  v.release(getContext().currentTime);
  voices.delete(key);
}

export function allOff() {
  for (const key of [...voices.keys()]) noteOff(key);
}

export function liveVoices() {
  return [...voices.values()];
}

Voice stealing matters more than it looks. A gamepad has enough buttons that an enthusiastic player can hold eight notes while mashing a ninth, and without a cap you accumulate oscillators until the graph gets expensive. Stealing the oldest is the simplest policy that sounds acceptable.

How do you turn a stick into a smooth parameter?

Feed the polled value into setTargetAtTime with a short time constant, instead of assigning to .value.

This is the single most important technique in the post. Your poll loop runs at 60 Hz. The audio graph runs at 44,100 or 48,000 samples per second. If you assign filter.frequency.value = x sixty times a second, you are producing a staircase with sixty discontinuities per second, and every one of them is a click. setTargetAtTime instead starts an exponential approach toward the target, interpolated at audio rate, and completely removes the staircase.

// audio/smooth.js
import { getContext } from './context.js';

// Time constant: smaller = snappier, larger = smoother. 0.01-0.05 feels right
// for a performance control at a 60 Hz poll rate.
export function glide(param, target, tau = 0.02) {
  const ctx = getContext();
  param.setTargetAtTime(target, ctx.currentTime, tau);
}

// Map a -1..1 axis onto an exponential frequency range, which is how
// cutoff should feel to a human hand.
export function axisToFreq(axis, { min = 120, max = 9000 } = {}) {
  const unit = (axis + 1) / 2;                       // 0..1
  return min * Math.pow(max / min, unit);            // exponential, not linear
}

The exponential mapping is the other half of making a stick feel right. Pitch and filter cutoff are perceived logarithmically, so a linear sweep from 120 Hz to 9 kHz spends almost all of its travel in a range your ear treats as "bright", and the musically useful bottom half happens in the first few millimetres of stick movement. Mapping exponentially distributes the interesting region across the whole throw.

Putting it together: the poll loop

One requestAnimationFrame loop that diffs buttons for notes and glides axes into parameters.

// instrument.js
import { getContext, unlock } from './audio/context.js';
import { noteOn, noteOff, allOff, liveVoices } from './audio/polyphony.js';
import { glide, axisToFreq } from './audio/smooth.js';

const SCALE = [0, 2, 4, 7, 9, 12, 14, 16];   // major pentatonic-ish, two octaves
const ROOT = 48;                             // C3

const prevButtons = [];

export function startInstrument(padIndex = 0) {
  function frame() {
    requestAnimationFrame(frame);

    const ctx = getContext();
    if (ctx.state !== 'running') return;

    const pad = navigator.getGamepads()[padIndex];
    if (!pad || !pad.connected) return;

    // Buttons 0-3 and 12-15 become eight scale degrees.
    const noteButtons = [0, 1, 2, 3, 12, 13, 14, 15];
    noteButtons.forEach((b, i) => {
      const btn = pad.buttons[b];
      if (!btn) return;
      const down = btn.pressed;
      if (down === prevButtons[b]) return;
      prevButtons[b] = down;

      if (down) {
        const vel = 0.4 + 0.6 * (pad.buttons[7]?.value ?? 0);   // R2 as velocity
        noteOn(b, ROOT + SCALE[i], vel);
      } else {
        noteOff(b);
      }
    });

    // Left stick X -> filter cutoff on every live voice.
    const cutoff = axisToFreq(pad.axes[0] ?? 0);
    for (const v of liveVoices()) glide(v.filter.frequency, cutoff, 0.02);

    // Left stick Y (inverted) -> resonance.
    const q = 1 + 14 * ((-(pad.axes[1] ?? 0) + 1) / 2);
    for (const v of liveVoices()) glide(v.filter.Q, q, 0.03);

    // L2 -> master-ish detune on every oscillator.
    const detuneCents = (pad.buttons[6]?.value ?? 0) * 25;
    for (const v of liveVoices()) glide(v.osc.detune, detuneCents, 0.01);
  }

  requestAnimationFrame(frame);
}

document.getElementById('start').addEventListener('click', async () => {
  await unlock();
  startInstrument(0);
});

document.addEventListener('visibilitychange', () => {
  if (document.hidden) allOff();
});

That is a complete instrument in about seventy lines, with velocity from an analogue trigger, a resonant filter on two stick axes, and detune on a second trigger. Everything that touches an AudioParam goes through glide(); nothing assigns .value directly.

The visibilitychange handler is not optional. requestAnimationFrame does not run in a hidden tab, so a note held when you switch tabs will never see its release and will sustain forever. The same problem and the same fix appear in the Gamepad API polling guide.

Where does the browser's latency actually come from?

From four stages, two of which the Web Audio API will tell you about directly.

Rather than quoting millisecond figures from somewhere, measure the stages you can and reason about the rest:

1. Device to OS. The controller's USB or Bluetooth report interval. Not visible from JavaScript; Bluetooth is structurally worse than USB. Covered in Bluetooth MIDI latency.
2. OS to your poll. Bounded by your loop rate. At 60 Hz rAF, the worst case is one frame of staleness. gamepad.timestamp tells you when the data actually arrived, so you can see this stage.
3. Graph to audio subsystem. This is audioContext.baseLatency, in seconds, and you can read it.
4. Subsystem to speaker. This is audioContext.outputLatency, in seconds, estimated by the browser. Bluetooth headphones move this number dramatically.

The useful consequence is that two of the four stages are measurable at runtime on the exact machine the user is on, which beats any benchmark table. Print them. If a user says it feels laggy, ask for those two numbers before you theorise.

// Show the real cost on this machine, in milliseconds.
function showLatency() {
  const c = getContext();
  const base = (c.baseLatency ?? 0) * 1000;
  const out  = (c.outputLatency ?? 0) * 1000;
  document.getElementById('latency').textContent =
    `base ${base.toFixed(1)} ms + output ${out.toFixed(1)} ms @ ${c.sampleRate} Hz`;
}

When do you need an AudioWorklet?

When you need DSP the built-in nodes cannot express — not merely because you want things to be faster.

A common misconception is that oscillators and filters run on the main thread and an AudioWorklet moves them to the audio thread. They are already on the audio rendering thread; the built-in node types are implemented natively. What an AudioWorklet buys you is the ability to write your own per-sample processing in JavaScript or WebAssembly and have it run on that same thread.

So reach for one when you want a custom waveshaper, a physical model, a bespoke filter topology, or sample-accurate logic that the node graph cannot express. For an oscillator, a filter and an envelope, it adds complexity and nothing else. MDN's AudioWorklet guide is the starting point when you do need it.

If what you actually want is a richer synthesis toolkit rather than custom DSP, a library is the better answer. Tone.js with a gamepad, Faust in the browser and p5.sound each sit at a different point on that curve.

What are the limits of a tab instrument?

It is self-contained, which is both its entire advantage and its entire limitation.

You cannot record it into a DAW without routing audio through a loopback device. You cannot use it as a controller for anything else. It stops when the tab is hidden. It has no plugin hosting, no sample library, no automation lanes. And the audio chain runs through the browser's buffer rather than a dedicated audio driver, which is a real cost on Windows where ASIO exists and the browser does not use it.

What it is unbeatable at: handing someone a link and having them playing in four seconds, on any device including an iPad, with nothing installed. That is a genuinely rare property and it is worth building for on its own terms rather than treating it as a lesser DAW. The visualiser is the same idea applied to MIDI input.

When you want the controller to drive your actual sessions — your plugins, your sampler, your mixer — the controller has to leave the tab. That is the whole argument in browser versus native MIDI.

FAQ

Can you build a playable instrument with just the Gamepad API and Web Audio?

Yes, and with no MIDI involved at all. The Gamepad API reads the controller, the Web Audio API synthesises the sound, and both are supported far more widely than Web MIDI, including on Safari and iPad.

Why is there no sound until I click something?

Browser autoplay policy creates the AudioContext in a suspended state. You must call resume() from inside a user gesture handler. A gamepad button press does not count as a user gesture for this purpose, so you need one real click or keypress first.

Why does changing an AudioParam value directly cause clicks?

Because assigning to the value property is an instantaneous jump and a discontinuity in the waveform is audible as a click. Use setTargetAtTime or a ramp method so the change is interpolated across samples instead.

Can I reuse an OscillatorNode for the next note?

No. An OscillatorNode can be started only once and cannot be restarted after it stops. Create a fresh node per voice and let it be garbage collected after its stop time, which is the intended pattern.

What do baseLatency and outputLatency tell me?

baseLatency is the seconds of latency incurred passing a buffer from the end of your audio graph into the host audio subsystem. outputLatency estimates the seconds between the browser handing that buffer over and the first sample reaching the output device. Together they are the browser audio output cost, measured rather than guessed.

Does latencyHint actually do anything?

It is a request, not a guarantee. MDN is explicit that the browser may ignore it. You can read back the effect on baseLatency, where interactive typically reports a much smaller value than playback.

When should I use an AudioWorklet instead of built-in nodes?

When you need custom DSP that the built-in node types cannot express, or when you want your processing to run on the audio rendering thread instead of the main thread. For an oscillator, a filter and an envelope, the built-in nodes are already on the audio thread and an AudioWorklet adds nothing.

Does a gamepad instrument work on an iPad?

Yes. Both the Gamepad API and the Web Audio API are supported in Safari on iOS and iPadOS, where Web MIDI is not. A self-contained gamepad instrument is the one browser music rig that works across the whole Apple ecosystem.

Build it tonight, outgrow it next week

Paste the five modules above into a folder, serve them from localhost, click start, press a face button and you have an instrument. It will work on a Mac in Safari and on an iPad, which is more than any Web MIDI rig can say — the support comparison is in Web MIDI browser support in 2026.

The moment you want that controller driving Ableton, Logic, a hardware sampler or all three at once, it has to leave the tab. Universal Controller MIDI publishes a native virtual MIDI port from a DualSense, Xbox, Switch Pro or 8BitDo pad on macOS, Windows and Linux — free for one controller, one port and three saved presets, AUD $89 one-time for Pro.