Xbox One Occupancy Quick Reference

Martin Fuller, Xbox Advanced Technology Group

Published May 13th 2016

In this topic

Overview of a compute unit

Local data store

Cheat sheets

Bank conflicts

Summary

Related articles

Overview of a compute unit

The Xbox One GPU is comprised of two shader engines, each with six compute units (CUs).

• Each CU is comprised of:
  • Four SIMD units, each processing waves 64 threads wide.
  • 64 KB of LDS, allocated in multiples of 256 bytes.
  • 16 KB of four-way set associative DCache and a number of other units not relevant to occupancy considerations (such as the TA and SQ).
• Each SIMD:
  • Can host up to 10 waves, depending on available resources (registers and LDS).
  • Has 256 vector registers (VGPRs) and 512 scalar registers (SGPRs).
    • VGPRs are 64 elements wide, one per thread.
    • SGPRs hold thread invariant data, including descriptors.
    • Some SGPRs are reserved, and the maximum the user can require is 104.
    • VGPRs are allocated in multiples of four, SGPRs in multiples of eight.
  • Requires a minimum of four cycles to complete each instruction.
• Each CU can execute a maximum of five instructions per clock; each must be a different type and issued to a different wave on the same SIMD. The seven types are:
  • Vector ALU
  • Vector memory operation
  • Scalar ALU or scalar load
  • Branch
  • Any two of
    • Export
    • GDS (read/write/atomic)
    • LDS (read/write/atomic)
  • Specials (nop, sleep, waitcnt, barrier, setprio, setvskip, halt)

Because the fastest instruction takes a SIMD four cycles to complete, and there are four SIMDs per CU, each CU can issue a vector ALU operation per cycle. Waves per SIMD is known as occupancy. Typically, engineers should aim for an occupancy of at least 3, preferably 4 or greater.

Local data store

When a shader uses LDS, all waves in a threadgroup must execute on the same CU. Although each CU has 64 KB of LDS, we prohibit allocating more than 32 KB per threadgroup.

In the event that there is just one wave per threadgroup (<= 64 threads issued) and that group requires 32 KB of LDS, the CU is only able to run two waves concurrently, meaning that half the SIMDs are unused. This is a catastrophic performance issue.

Cheat sheets

LDS per threadgroup (KB, aligned to 256 bytes)

The maximum LDS allocation in bytes for each occupancy tipping point is given by:

  floor(((65536 / (maxWavesPerSIMD * 4)) * ThreadsPerGroup) / 256) * 256

Bank conflicts

LDS is arranged into 32 banks of 2 KB each. Each bank has a 32-bit wide bus, providing a maximum bandwidth of 128 bytes per cycle per CU. An LDS instruction is split into two operations, one for the low 32 threads and one for the high 32. For conflicts, we only need to consider one of these 32-bit wide accesses.

Of the 16-bit address, the most significant 9 bits select the address within the bank, the next 5 bits select the bank, and the least significant 2 bits select the byte within the 32-bit wide address. (However, currently we only expose 32-bit access.)

A bank conflict is generated when:

• Two or more threads in the same high or low 32 threads access the same bank.
• Except in the case of reads, a bank conflict may be avoided only when all reads from the same bank access the same 32 bits, in which case the value is broadcast.

In the case of a bank conflict, the memory operation will be N times slower, where N is the highest number of threads accessing the same bank. For example:

• The worst case occurs when writing to LDS with a per thread stride of 32 floats/dwords. This causes every thread to access the same bank and results in a 32x slowdown.
• With a stride of just 2 floats, half the threads will hit on the same bank, causing a 2x slowdown.
• Not all strides > 1 are bad however; for example, the following stride of three generates no conflicts:
  • The first 11 threads will access banks 0,3,6,9,12. . . 30
  • The second 11 threads will access banks 1,4,7,10,13. . .31
  • The third 10 threads will access banks 2,5,8,11,14. . . 29

The advice is therefore to avoid writing to LDS with strides that are a power of 2 .

There is no penalty if threads in the high 32 threads access the same banks as threads in the low 32 threads.

1. Greater than zero of course

Summary

This paper provides a quick reference that you can use when optimizing for occupancy. While register allocation is relatively straight forward, effective LDS optimization is dependent not only on occupancy, but also on ensuring that waves do not generate bank conflicts.

Related articles

• Xbox One GPU Overview white paper
• Xbox One GPU Compute Units white paper