Transientica
A rhythm game you play by beatboxing. I am developing it solo in Unity 6 and wrote its audio recognition in C#.
The game learns how each player makes a kick, a snare and a hi‑hat, then reads every live hit as an input on one of three lanes.
2025 to now. Around the recognition I built the rest of the game: song selection, beatmaps, scoring and saved results. The gameplay on this page is from the March 2026 build.
Trained on the player’s own kick, snare and hi‑hat
Before the first song, the player records at least ten kicks, snares and hi‑hats with their own microphone. From each recording the game measures seven features (spectral centroid, rolloff, flux, zero‑crossing rate, RMS, attack time and spectral flatness) and saves them, labelled by sound, in a JSON file, so a player can retrain between rounds. Training tests itself with leave‑one‑out cross‑validation and asks for clearer recordings below 70% accuracy.
During a song, every hit the microphone registers goes through a 512‑sample FFT and gets the same seven features. Its five nearest training recordings vote, weighted by distance, and the game shows the winning sound at the top of the screen with the classifier’s confidence under it.
The FFT and the k‑nearest‑neighbours (KNN) classifier are my own C# code, so the recognition runs inside the game with no Python beside it.
Judged on the audio clock
Each hit is timed on Unity’s audio clock (dspTime), moved back half a 512‑sample FFT window (about 5.8 ms at 44.1 kHz) for the analysis delay, and matched to the nearest note: within 60 ms is Perfect, 130 ms Great, 200 ms Good.
For the microphone delay, a calibration plays clicks at 100 BPM while the player beatboxes along, and the median gap becomes the offset, adjustable afterwards in 1 ms steps.
The 2025 Python prototype
The first version was a Python script. It found each hit with onset detection, classified it with a machine learning model I had trained on three beatbox sounds (a boom for the kick, a sharp ch for the snare, a crisp ts for the hi‑hat) and sent the result to Unity over OSC, where the lane for that sound reacted.