2025–present · Rhythm game in development

Transientica

A rhythm game I am developing solo in Unity. I wrote the C# audio recognition system that turns the player’s beatboxing into kick, snare and hi-hat inputs.

Unity 6 · C# · KNN · FFT

Gameplay demo

Transientica gameplay: three lanes labelled KICK, SNARE and HI-HAT, a note falling towards the line at the bottom, a score and a combo counter

Training with the player’s voice

Players record at least ten examples of each beatbox sound with their own microphone. The classifier learns from these examples, then identifies live hits during a song.

The game compares each hit with the beatmap and scores its timing.

Training and gameplay in the current C# build.

Audio recognition and note timing

The current build captures, analyses and scores the microphone input inside Unity.

The three empty lanes at the start of a song, labelled KICK, SNARE and HI-HAT, with the score at zero and a MIC ON meter at the right edge

01

Microphone input

One lane each for kick, snare and hi-hat, with the microphone open before the first note.

A hit named SNARE across the top of the screen with a confidence figure under it, and the upcoming notes listed in a strip below

02

Sound classification

An onset triggers a 512-sample FFT and seven features. A KNN classifier compares them with the player’s own recordings.

A hi-hat note inside its ring on the way to the line, with the judgement and combo count beside it and sparks where the last note landed

03

Timing and calibration

The hit is matched to the nearest note on Unity’s audio clock. A short calibration estimates the microphone delay, adjustable in 1 ms steps.

Moving recognition from Python to C#

The first version used a separate Python classifier and sent its results to Unity over OSC. I replaced it with a KNN classifier and FFT written in C#, keeping the audio processing and gameplay in one application.

The 2025 prototype, with its Python classifier.

Song selection and saved training

The current build includes song selection, beatmaps, scoring and saved results. Player recordings are stored as labelled features in JSON, so training can be updated between rounds.

The C# rebuild has not yet had a like-for-like accuracy and latency benchmark against the earlier Python prototype.

The track select screen with START and BACK buttons, arrows for moving between tracks, and a panel naming the chosen song with its tempo and note count

Choosing a beatmap in the current build.