Audio artifacts can come in many varieties. Diagnosing when the sound output by a title diverges from expectation can often reach into multiple non-audio areas of a title. A number of existing tools and diagnostics, along with a descriptive language for evaluating and quantifying the source of artifacts, can be used to diagnose and resolve audio glitching issues.
For the purpose of this topic, a glitch is defined as any unexpected audio output from the console. Glitches generally fall into one of a number of categories, and can occur for the master output, a submix, or an individual voice:
Observation alone can often provide clues as to potential issues; the quality, cadence, and repeatability of an audio disruption can also assist with diagnosis.
The following table shows some glitch types and possible diagnoses.
| Glitch type | Measurement | Possible diagnosis |
|---|---|---|
| Dropout | Duration | If duration correlates with audio engine frame size, engine was likely starved. |
| Period (frequency of dropout) | If intermittent, look at CPU conditions at time of dropout by using performance tools such as PIX or Windows Performance Analyzer (did another process consume too many cycles of the CPU?). If consistent, look for constant load causes, such as contention within the thread scheduler. | |
| Stutter | Period (amount of audio data that repeats) | If correlated with audio engine frame size, may indicate engine starvation (some engines will resubmit last known buffers), or the use of a circular buffer whose read cursor has overrun a stalled write cursor. |
| Overall duration | If one time, audio may have framed out. If sustained, audio may have been actively starved by other blocking processes. | |
| Pop | Period/frequency | If occurring at a predictable frequency, may indicate memory corruption, DSP implementation issues, or source content artifacts. |
| Warble | Quality | If well-correlated with audio amplitude, may indicate clipping, saturation, or rounding/approximation issues within a DSP effect. |
Diagnosing a glitch often can be assisted by isolating the point in the pipeline where the artifact was introduced; a typical first step is soloing individual mixes and voices to determine whether the noise is related to an individual item versus the overall mix. Glitches on individual voices (or submixes) can often indicate issues with audio content or processing rather than engine-wide failures seen at a master output.
Duration and periodicity can give hints as to the process that has been starved or otherwise disrupted. Often an entire frame (set of sequential samples, often across the number of channels supported by an output endpoint) of audio has failed to render, so correlating the frame size to the size of the disruption can provide clues as to the source of the failure. The following table represents default values for commonly used audio systems.
| Interface | Frame size (bold denotes fixed) |
|---|---|
| Wwise | 512 samples (10 2/3 msec) |
| FMOD | 512 samples (10 2/3 msec) |
| XAudio2 | 256 samples (5 1/3 msec) |
| SHAPE audio hardware processing (including XMA decode) | 128 samples (2 2/3 msec @ 48 KHz) |
| WASAPI render endpoint (XDK) | ~5 msec |
| WASAPI render endpoint (ADK) | ~10 msec |
| WASAPI capture (i.e. microphone) | query IAudioClient::GetDevicePeriod |
| Game Chat 2 (render/capture) | 40 msec |
Frame size, whether in samples or in milliseconds, typically requires conversion to reflect frame size in bytes—be aware of the client format (on Xbox One, most typically 7.1 channel 48 KHz 16-bit integer) and any block alignment requirements of that format.
Glitching is often associated with the audio renderer not receiving enough time to complete processing for a frame. This may be because other non-audio processing consumed more resources, and/or that the audio thread did not wake up in time to create a full frame of audio data before the renderer needed it. Tools such as PIX allow for real-time capture of both CPU metrics and actual rendered audio to help correlate such issues.
Adjusting thread affinity or priority for audio and/or other processing is often a potential solution. Several guidelines for audio thread management follow:
Most titles achieve these guidelines by locking audio processing to a single core (and not to logical core 6).
Glitches can also be diagnosed to originate from a number of other sources wholly within the realm of the audio engine. Most notably, artifacts can be present in the source content (so the rendering is actually “correct,” but the content itself contains pops, warbles, or dropouts); examples include:
Real-time applied digital signal processing can also introduce audible distortion, whether via clipping/saturation (attempting to represent a signal larger than can be represented), zippering (title- or implementation-driven quantization of parameters leading to audible state changes), or inadvertent misconfiguration/replication (for example, accidentally triggering a sound twice).
Glitch detection in an automated manner can allow studios to review whether the fault lies in the audio system (too much processing attempted in a frame) or other title components (consumed more than expected CPU, not allowing audio time to process). Both XAudio2 and WASAPI support queriable properties (and can also optionally spew debug text strings via a connected PC) to aid in such detection. Additionally, title developers can implement analysis of output audio streams to determine additional glitch scenarios (see the ‘resources’ section at the end of this topic for a GDC presentation on the topic).
For a capture client (such as a microphone), dropouts due to starvation can be detected by looking at the “flags” parameter returned within IAudioCaptureClient::GetBuffer.
For a render client (such as speakers), starvation occurs if the engine needs to read from it and there is less than one period of audio data available (as mentioned earlier, 10 milliseconds). WASAPI by default will at least double buffer (allocate 20 milliseconds) the render buffer, although titles and WASAPI engine instances can request even larger buffers if desired—at the expense of greater latency. For background media playback, this may not be an issue. But for games, typically the minimum buffering is used. Applications can indeed query how much audio data is available, and determine that starvation might occur if the amount falls below 10 msec. (This is not definitive because the audio engine may not yet need data, so the title may still have time to fill the buffer.)
To more definitively detect starvation, Windows Performance Analyzer (WPA) can be used to view audio output and determine sources of audio glitches. Causing a glitch manually to test glitch detection systems can be performed by emitting debug spew or running high priority processing on the same CPU as the audio system.
The Event Tracing for Windows (ETW) system can submit relevant events; subscribe to provider Microsoft-Windows-Audio (guid AE4BD3BE-F36F-45b6-8D21-BDD6FB832853), which emits relevant events, such as EVT_GLITCH_CP_SERVER_INPUT_STARVATION.
XAudio2’s debugging capabilities automatically report (via debug spew) and can be queried for glitch and starvation issues. For more details, see Debugging Audio Glitches in XAudio2 on MSDN.
Audio debugging can often be more qualitative than quantitative, but there are tools for the latter, which this paper intended to highlight. Glitches can be caused by the audio engine itself, other processes, source content, or implementation. Awareness of these scenarios and the analysis steps and thread management best practices can help you more readily diagnose and resolve issues of objectionable audio artifacts in production titles.