p5.js Generative Audio — Gamepad WebMIDI Patch

p5js gamepad audio is one of those rigs that sounds esoteric until you have it running, then you wonder why you ever used anything else. p5.js gives you canvas, sound, and a 100-line learning curve. A gamepad gives you eight continuous controls and a fistful of buttons. Universal Controller MIDI glues them together via WebMIDI. The result is a single browser tab that plays generative drones, paints reactive visuals, and runs offline. No build chain. No npm install.

Why route the gamepad through MIDI

You can absolutely call navigator.getGamepads() from p5 and read sticks directly. But raw gamepad data is messy — deadzones differ, sticks drift, button repeat is platform-specific. The bridge normalises all of that into clean 0..127 MIDI CCs. The same sketch then works with a DualSense today and a Switch Pro Controller tomorrow without touching code. The bridge handles the hardware; p5 handles the sound and the pixels.

The minimum viable rig

Drop this into the p5 editor. It's the smallest patch that actually makes noise — one oscillator, frequency mapped to stick X, amplitude mapped to right trigger.

let osc, env;
const cc = new Array(128).fill(0);

async function setup() {
  createCanvas(640, 360);

  // WebMIDI bootstrap. p5 doesn't help with this; vanilla browser API.
  const midi = await navigator.requestMIDIAccess();
  for (const input of midi.inputs.values()) {
    input.onmidimessage = (e) => {
      const [status, d1, d2] = e.data;
      if ((status & 0xf0) === 0xb0) cc[d1] = d2 / 127;
    };
  }

  osc = new p5.Oscillator('sine');
  osc.start();
  osc.amp(0);
}

function draw() {
  background(10);

  // Stick X (CC 0) → freq 80..880Hz. Trigger R2 (CC 7) → amp 0..0.4.
  const freq = 80 + cc[0] * 800;
  const amp = cc[7] * 0.4;

  osc.freq(freq, 0.05);
  osc.amp(amp, 0.05);

  // Cheap visual feedback.
  noStroke();
  fill(94, 234, 212, 200);
  const r = 40 + amp * 200;
  ellipse(width / 2, height / 2, r, r);
}

Hit Run, wiggle the stick, squeeze R2. You have a playable instrument. Note the 0.05 ramp times — without those, every CC step would click. p5 doesn't auto-smooth; you do it explicitly.

Adding a filter and an LFO

A single oscillator gets boring in ten seconds. Add a low-pass filter on stick Y, modulate the cutoff with an LFO whose rate is left trigger.

let osc, filt, lfo;
const cc = new Array(128).fill(0);

async function setup() {
  createCanvas(640, 360);
  await initMIDI();

  osc = new p5.Oscillator('sawtooth');
  filt = new p5.LowPass();
  lfo = new p5.Oscillator('sine');

  osc.disconnect();
  osc.connect(filt);
  filt.process(osc);

  lfo.disconnect();
  lfo.amp(400);          // cutoff modulation depth
  lfo.start();

  osc.start();
  osc.amp(0);
}

function draw() {
  background(10);

  const freq   = 60 + cc[0] * 600;     // L-stick X
  const cutoff = 200 + cc[1] * 4000;   // L-stick Y (already 0..1 from bridge)
  const amp    = cc[7] * 0.35;         // R2 trigger
  const lfoRate = 0.1 + cc[6] * 12;    // L2 trigger

  osc.freq(freq, 0.04);
  osc.amp(amp, 0.04);
  filt.freq(cutoff + lfo.getAmplitude() * 400);
  filt.res(2 + cc[2] * 18);            // R-stick X → resonance
  lfo.freq(lfoRate, 0.1);

  drawScope();
}

function drawScope() {
  // Scrolling waveform tied to CC values — reuse the same CC array.
  stroke(94, 234, 212, 180);
  noFill();
  beginShape();
  for (let i = 0; i < width; i += 4) {
    const y = height / 2 + sin(i * 0.04 + frameCount * 0.05) * (cc[7] * 100);
    vertex(i, y);
  }
  endShape();
}

Visuals and audio share the same CC source. That's the whole trick — there's only one truth (the gamepad), and both subsystems read from it. No syncing required.

CC map for this sketch

Gamepad input	CC	Maps to
Left stick X	CC 0	Oscillator frequency 60–660 Hz
Left stick Y	CC 1	Filter cutoff 200–4200 Hz
Right stick X	CC 2	Filter resonance 2–20
L2 trigger	CC 6	LFO rate 0.1–12 Hz
R2 trigger	CC 7	Master amp 0–0.35
Face buttons	Note 36–39	Trigger envelope, change waveform

Polyphony with note triggers

Buttons land as MIDI notes. p5.sound doesn't have a built-in voice manager, but a pool of oscillators is six lines.

const voices = [];
const POOL = 6;

function initVoices() {
  for (let i = 0; i < POOL; i++) {
    const v = new p5.Oscillator('triangle');
    const e = new p5.Envelope(0.005, 0.4, 0.0, 0.6);
    v.start(); v.amp(0);
    voices.push({ osc: v, env: e, free: true });
  }
}

function trigger(note) {
  const v = voices.find(x => x.free) ?? voices[0];
  v.free = false;
  v.osc.freq(midiToFreq(note));
  v.env.play(v.osc);
  setTimeout(() => v.free = true, 1100);
}

// Wire it up: listen for Note On in your MIDI handler.
// if ((status & 0xf0) === 0x90 && d2 > 0) trigger(d1);

Six voices, round-robin allocation. Enough for chord stabs over the stick-driven drone. For serious polyphony move to Tone.js — it has a real PolySynth with proper voice stealing.

Tradeoffs and where p5 breaks down

p5.sound is great for sketches and live performance up to a point. It has no MIDI clock sync, no plugin host, no MPE, no precise scheduling beyond setTimeout. If you need tempo-locked sequencing, the p5.sound reference tops out around groove-machine complexity. For anything beyond that, use this sketch as the visualiser and let Ableton or Tone.js handle the audio. For raw modulation depth across multiple synths, the sound design gamepad guide covers gamepad as an LFO source.

The browser is a more capable instrument than people give it credit for. Plug a gamepad in, paste this sketch, and you have a noise machine on every device with Chrome.