Research 2024–2025 · thesis published 2026 · Howest DAE

Testing whether AI-generated sound is fast enough for gameplay

A thesis and working Unreal prototype that measures how long a generated sound takes to reach the player after an in-game event.

As the sole researcher and developer, I built the Unreal client, Python generation server, cache and fallback. The tested AI generators missed the interaction budget, so the system prepares likely sounds early and uses simple synthesis when one is not ready.

Solo researcher and prototype developer · Unreal Engine 5 · Python · 68-page thesis

See the timing test Read the thesis

Timing test

The tested AI generators took seconds, not milliseconds

I timed five methods (one procedural baseline and four AI configurations) from an in-game event to the first audible sample. Each method and prompt combination was repeated five times. A separate, single-rater quality check covered 60 anonymised samples.

Upper limit used for interactive SFX150 ms
95th-percentile time from the gameplay event to the first audible sample. After reviewing prior latency studies, the thesis uses a 100–150 ms budget for interactive SFX; this chart marks the upper limit.

The test machine used an AMD Ryzen 9 7940HS, an RTX 5070 Laptop GPU with 8 GB VRAM, 64 GB RAM, Unreal Engine 5.4.4 and Python 3.11.7.

The figures follow the final test method documented in the thesis. They cover the full wait experienced in the prototype (request, generation or transfer, decoding and playback), not only the model itself.

Playback strategy

Prepare likely sounds before they are needed

Because first-time generation is too slow, the server watches game state and prepares likely sounds before the event occurs.

Approaching a door can prepare door sounds; entering a forest can prepare ambience. The result is stored so the game can play it later without making the player wait.

If the prediction is wrong, the game plays a simple synthetic fallback while the generated sample is prepared for a later event.

Playback path Predict → generate and cache → play on demand → use synthesis if the cache misses.

Evidence

Audio samples and the running prototype

Use the same prompt across each method to hear the difference. These are selected examples; the full results and test method are in the thesis.

Procedural DSP Heavy footstep on wet stone

Low latency, but clearly synthetic.

Latency 15-25ms Role Fallback
Unreal client · in-game requests
Python server · models, cache and retries