Processing Java + OSC — Gamepad-Driven Visual Rig

Processing gamepad osc is the rig you build when you've outgrown the browser. Casey Reas and Ben Fry's Processing has thirty years of generative-art history, a thousand libraries, and the largest visual-coding community outside Houdini. It's also Java, which means there's no clean native gamepad library worth using. The fix is routing the controller through Universal Controller MIDI → OSC → oscP5. Three protocols, one cable, full control.

Why OSC, not MIDI, into Processing

Java's MIDI stack is a known mess on macOS — javax.sound.midi has its own opinions about virtual ports, and Processing's MidiBus library wraps that with limited maintenance. OSC is the way out. UDP packets are simple, the bridge ships OSC out of the box, and oscP5 is the de facto library for it. You also get free networking: run Processing on a Mac mini behind the stage, the bridge on a laptop with the controller, both talking over the venue Wi-Fi. OSC vs MIDI for gamepads goes deep on the tradeoffs.

Bridge setup — turn on OSC out

In the bridge: Settings → OSC. Toggle Enabled, set host 127.0.0.1, port 9000, namespace /gamepad. The bridge will now send one OSC packet per CC change:

# Example outgoing packets when you wiggle a stick + click a button
/gamepad/cc/0   0.624      # L-stick X, float 0..1
/gamepad/cc/1   0.500
/gamepad/cc/7   0.882      # R2 trigger
/gamepad/note/36 1         # Cross pressed
/gamepad/note/36 0         # Cross released

Floats by default. Toggle to ints if you'd rather work in 0..127 — but for visuals, normalised floats save you a division on every frame.

The Processing skeleton

Install oscP5 (Sketch → Import Library → Add Library). The sketch below mirrors the bridge's CC stream into a local float array and uses it to drive a rotating, noise-driven mesh.

import oscP5.*;
import netP5.*;

OscP5 osc;
float[] cc = new float[128];

void setup() {
  size(1280, 720, P3D);
  // Listen on UDP 9000 — matches the bridge's OSC output port.
  osc = new OscP5(this, 9000);
}

void draw() {
  background(8);
  translate(width / 2, height / 2);

  // L-stick X (CC 0) → rotation. R-stick Y (CC 3) → tilt.
  rotateY(map(cc[0], 0, 1, -PI, PI));
  rotateX(map(cc[3], 0, 1, -PI/2, PI/2));

  // L2 (CC 6) → mesh density. R2 (CC 7) → brightness.
  int density = (int) map(cc[6], 0, 1, 8, 96);
  stroke(94, 234, 212, (int)(cc[7] * 255));
  noFill();

  float r = 200;
  for (int i = 0; i < density; i++) {
    float a = TWO_PI * i / density;
    float n = noise(cos(a) + frameCount * 0.01, sin(a)) * 60;
    pushMatrix();
    rotate(a);
    line(0, 0, r + n, 0);
    popMatrix();
  }
}

// Single OSC handler — pattern-matches /gamepad/cc/N
void oscEvent(OscMessage m) {
  String addr = m.addrPattern();
  if (addr.startsWith("/gamepad/cc/")) {
    int n = Integer.parseInt(addr.substring(12));
    if (n >= 0 && n < 128) cc[n] = m.get(0).floatValue();
  }
}

Hit Run. Wiggle the sticks — the mesh rotates. Squeeze L2 — more spokes. The whole sketch is forty lines and it'll outpaint a $4000 plugin chain.

Handling notes for discrete events

CCs are continuous; notes are events. Face buttons should fire one-shot transitions, not crossfades. Add a second handler for the note namespace.

int activePalette = 0;
boolean flash = false;
float flashAmt = 0;

void oscEvent(OscMessage m) {
  String addr = m.addrPattern();
  if (addr.startsWith("/gamepad/cc/")) {
    int n = Integer.parseInt(addr.substring(12));
    cc[n] = m.get(0).floatValue();
  } else if (addr.startsWith("/gamepad/note/")) {
    int note = Integer.parseInt(addr.substring(14));
    int vel  = m.get(0).intValue();
    if (vel > 0) onButton(note);
  }
}

void onButton(int note) {
  if (note == 36) activePalette = (activePalette + 1) % 4;   // ✕ cycle palette
  if (note == 37) flashAmt = 1.0;                            // ◯ strobe
  if (note == 38) saveFrame("####.png");                     // □ screenshot
}

void draw() {
  // ... existing draw code ...
  if (flashAmt > 0) {
    fill(255, flashAmt * 255);
    rect(-width, -height, width * 2, height * 2);
    flashAmt *= 0.85;
  }
}

Default OSC namespace

Gamepad input	OSC address	Type
Left stick X / Y	/gamepad/cc/0 / cc/1	float 0..1
Right stick X / Y	/gamepad/cc/2 / cc/3	float 0..1
L2 / R2 triggers	/gamepad/cc/6 / cc/7	float 0..1
Touchpad X / Y	/gamepad/cc/16 / cc/17	float 0..1
Face buttons (✕◯□△)	/gamepad/note/36..39	int 0/127
D-pad	/gamepad/note/40..43	int 0/127
Rumble feedback	/gamepad/rumble	float 0..1 (inbound)

Sending OSC back for haptics

The bridge accepts OSC on a configurable port for haptic feedback. Fire /gamepad/rumble with a 0..1 float and the controller's rumble motors kick on for ~200 ms. Use it for downbeat feedback in long generative sets.

OscP5 oscOut;
NetAddress bridgeAddr;

void setup() {
  size(1280, 720, P3D);
  osc = new OscP5(this, 9000);
  oscOut = new OscP5(this, 9001);
  bridgeAddr = new NetAddress("127.0.0.1", 9100); // bridge inbound port
}

void onBeat() {
  OscMessage rumble = new OscMessage("/gamepad/rumble");
  rumble.add(0.8);
  oscOut.send(rumble, bridgeAddr);
}

When Processing is the right home

Processing wins for sketch-grade generative art, performance pieces, and installations where you want one self-contained executable. It loses to TouchDesigner for video-heavy work and to Hydra for live-coded VJ sets where the audience watches the code. The Processing libraries page is the long tail — physics, computer vision, fabrication, all available once you have the gamepad piped in.

For Java-based interactive theatre, Isadora pairs well — see the Isadora gamepad guide. For OSC routing inside a DAW, Reaper + OSC is the canonical setup. Everything talks the same protocol; the bridge is the universal translator.

Universal Controller MIDI ships with OSC out the day you install it. Open Processing, paste the listener, you're patching.