Xbox One CPU: Introduction

The Xbox One CPU brings a host of modern micro-architectural performance features to console development. With Xbox One, a familiar instruction set architecture and high performance silicon mean developers can focus effort on content and features, not micro-optimization. The trend towards more parallel power continues in this hardware; so, an effective strategy for multi-core computing is more important than ever.

Architectural Overview

The Xbox One CPU is structured as two modules. A module contains four x64 cores, each running a single thread at 1.6 GHz. Each core contains a 32 KB instruction cache (I-cache) and a 32 KB data cache (D-cache), and the 4 cores in each module share a 2 MB level 2 (L2) cache. In total, the modules have 8 hardware threads and 4 MB of L2. The architecture is little-endian.

Figure 1.  Xbox One CPU layout.

Four cores communicate with the module’s L2 via the L2 Interface (L2I), and with the other module and the rest of the system (including main RAM) via the Core Communication Interface (CCI) and the North Bridge.

Caches

The caches can be summarized as shown in the following table.

Cache Policy Ways Set Size Line Size Sharing
L1 I Read only 2 256 64 bytes Dedicated to 1 core
L1 D Write-allocate, write-back 8 64 64 bytes Dedicated to 1 core
L2 Write-allocate, write-back, inclusive 16 2048 64 bytes Shared by module

The 4 MB of L2 cache is split into two parts, one in each module. On an L2 miss from one module, the hardware checks if the required line is resident in the other module-either in its L2 only, or any of its cores’ L1 caches. Checking and retrieving data from the other module’s caches is quicker than fetching it from main memory, but this is still much slower than fetching it from the local L1 or L2. This makes choice of core and module very important for processes that share data.

Memory access result Cycles Notes
L1 hit 3 Required line is in this core’s L1
L2 hit 17 Required line is in this module’s L2
Remote L2 hit, remote L1 miss 100 Required line is in the other module’s L2
Remote L2 hit, remote L1 hit 120 Required line is in the other module’s L2 & in remote core’s L1
Local L2 miss, remote L2 miss 144-160 Required line is not resident in any cache; load from memory

Both L1 and L2 caches have hardware prefetchers that automatically predict the next line required, based on the stream of load/store addresses generated so far. The prefetchers can derive negative and positive strides from multiple address sequences, and can make a considerable difference to performance. While the x64 instruction set has explicit cache control instructions, in many situations the prefetcher removes the need to manually insert these.

The Xbox One CPU does not support line or way locking in either L1 or L2, and has no L3 cache.

This document does not cover memory paging or translation lookaside buffers (TLBs) on the cores.

Instruction Set Architecture

The cores execute the x64 instruction set (also known as x86-64 or AMD64); this instruction set will be familiar to developers working on AMD or Intel based architectures, including that of desktop computers running Windows. x64 is a 64-bit extension to 32-bit x86, which is a complex instruction set computer (CISC) with register-memory, variable instruction length, and a long history of binary backward compatibility; that is, some instruction encodings have not changed since the 16-bit Intel 8086.

The x64 architecture requires SSE2 support, and Visual Studio makes exclusive use of SSE instructions for all floating-point operations. x64 deprecates older instruction sets: x87, Intel MMX®, and AMD 3DNow!®. x64 supports the following instruction set extensions:

The cores do not support XOP, AVX2, or FMA3/4 (fused multiply-add).

Architecturally, the cores each have sixteen 64-bit general purpose registers, eight 80-bit floating point registers, and sixteen 256-bit vector/SIMD registers. The 80-bit floating point registers are part of x87 legacy support.

Performance

Xbox One CPU cores run at 1.6 GHz; this is half the clock rate of the Xbox 360’s cores. Because of this, it is tempting to assume that the Xbox 360’s cores might outperform Xbox One’s cores. However, this is emphatically not true, for the reasons described in the following sections:

Sub-ISA Parallelism and Micro-Operations

Like most recent high-performance x64 processors, the cores do not execute the x64 instruction set natively; instead, internally instructions are decoded into micro-operations, which the processor executes. This translation provides opportunities to parallelize beyond traditional superscalar execution.

Xbox One CPU cores have dual x64 instruction decoders, so they can decode two instructions per cycle. On average, an x86 instruction is converted to 1.7 micro-operations, and many common x64 instructions are converted to 1 micro-operation. In the right conditions, the processor can simultaneously issue six micro-operations: a load, a store, two ALU, and two vector floating point. The core has corresponding pipelines: two identical 64-bit ALU pipelines, two 128-bit vector float pipelines (one with float multiply, one with float add), one load pipeline, and one store pipeline. A core can retire 2 micro-operations a cycle.

Out of Order Execution

Xbox 360 CPU cores execute in-order (also called program order) the instructions in exactly the order the compiler laid them out. The Xbox 360 CPU has no opportunity to anticipate and avoid stalls caused by dependencies in the incoming instruction stream, and no compiler can eliminate all possible pipeline issues.

In contrast, the Xbox One CPU cores execute fully out of order (OOO), also called data order, since execution order is determined by data dependencies. This means the processor is able, while executing a sequence of instructions, to re-order the micro-operations (not the x64 instructions) via an internal 64-entry re-order buffer (ROB). This improves performance by:

Register Renaming

A low count of registers can cause execution of instructions to be unnecessarily serialized. Similar in concept to translating x64 instructions to micro-operations, register names used in the x64 instruction stream are not used as is, but are instead renamed to point at entries in a large internal physical register file (PRF)—Xbox One cores have a 64-entry, 64-bit, general-purpose PRF and a 72-entry, 128-bit, vector float PRF. With renaming, the processor can disentangle serialization by register name alone, and to get better throughput, it can push independent micro-operations to earlier positions in the execution order via OOO.

Speculative Execution

Instruction streams can be regarded as being divided into basic blocks of non-branching code by branches. CPUs with deep pipelines execute basic blocks efficiently, but they face performance challenges around conditional branches. The simplest approach—stall until the conditional is determined and the branch direction is known—results in poor performance.

The Xbox One CPU is able to fetch ahead and predict through multiple conditional branches and hold multiple basic blocks in its re-order buffer, effectively executing ahead through the code from predicted branch outcomes. This is made possible via the core tracking which registers in the PRF represent speculative results—that is, those from basic blocks that are not currently certain to be executed. Once a branch direction is determined, if the core predicted the branch direction correctly, results from that basic block are marked as valid. If the core mispredicted, speculative results (which may include many basic blocks) are discarded, and fetching and execution then begins from the correct address.

Store Forwarding

With in-order execution, a store to memory followed shortly by a load from the same location can cause a stall while the contents of memory (usually via an L1 line) are updated; the stall ensures that the load gets the correct result, rather than a stale value. On Xbox 360, this commonly encountered penalty is called Load-Hit-Store. On Xbox One, the cores have store-forwarding hardware to deal with this situation. This hardware monitors the load store queue, looking for memory accesses with the same size and address; when it finds a match, it can short-cut the store and subsequent load via the physical register file, and thereby avoid significant pipeline stalls.

Highly Utilized Out of Order Load Store Engine

An Xbox One core is able to drive its load store unit at around 80-90% capacity, even on typical code, because the combination of OOO, register renaming, and store forwarding massively reduces pipeline flushes and stalls, permitting highly effective use of L1 bandwidth. This improvement is partly the result of the load store unit being able to reorder independent memory accesses to avoid data hazards: loads can be arbitrarily re-ordered, and stores may bypass loads, but stores cannot bypass other stores.

By contrast, the load store hardware in the Xbox 360 is utilized at about 15% capacity on typical code, due to the many pipeline bubbles from in-order execution on the cost-reduced PowerPC cores. In conjunction with pipeline issues, the major factors in the Xbox 360’s throughput being as low as 0.2 instructions per cycle (IPC) are L1 miss, L2 miss, and waiting for data from memory.

Cache Performance

The Xbox One CPU uses 64-byte cache lines, which makes a process less likely to waste bandwidth loading unneeded data. On Xbox 360, ensuring effective use of cache lines for 128-byte lines can be tricky. While an Xbox One core’s L1 data cache is the same size as on Xbox 360, it is not shared between two hyper threads, and it has better set associativity. L2 is effectively three times the size, for each hardware thread, and it has better associativity: 512 KB per hardware thread on Xbox One versus approximately 170 KB per hardware thread on Xbox 360. L1 and L2 bandwidth will be more efficiently utilized on an automatic basis via prefetching, smaller cache lines, register renaming, OOO, and store forwarding.

Advanced Branch Predictor

Effective branch prediction increases the likelihood that speculative execution will execute the right code path. The Xbox One CPU cores have an advanced dynamic branch predictor, able to predict up to 2 branches per cycle. Rather than a branch direction, an actual branch address is predicted, meaning the instruction fetch unit can speculatively fetch instructions without waiting for resolution of the branch instruction dependencies and the resultant target. The first-level sparse predictor stores information about the branch target for the first two branches in a cache line, hashed by line address in 4 KB of storage. The sparse information also indicates if more than 2 branches are present in that line, and indexes into a second-level dense predictor, by using a 4-KB set-associative cache of prediction information for branches in 8-byte chunks. A branch target address calculator checks relative branch predictions as early as possible in the pipeline to permit discarding incorrectly fetched instructions. In addition, the prediction unit contains a 16-entry call/return stack and a 32-entry out-of-page address predictor.

Areas of Optimization Focus

Xbox One will present a very different optimization target compared to Xbox 360; the processor is designed to run typical code at high IPC without major manual intervention. You will need to align your optimization efforts to accommodate the following conditions:

No More Top Penalties List

The Xbox 360 cores present multiple opportunities to hand-tune code, especially around major penalties resultant from in-order execution; further, these penalties are easily detected by examining the generated instruction stream alone. In contrast, Xbox One cores do not generate a similar set of penalties that can be easily removed from critical areas of processing.

Diminishing Returns of Hand Optimization

Game developers are usually highly skilled optimizers. However, anecdotal wisdom from multiple sources—including many optimization luminaries—says that hand-tuning x64 code on a core with the performance characteristics described earlier in this paper gives quickly diminishing returns and often slower code, for reasons that are not always intuitively discernible.

In general, Microsoft compilers generate code that uses SIMD instructions and the vector registers for floating point calculations—even scalar floating point—and Visual Studio 11’s compiler already auto-vectorizes, so improved SIMD performance is already in the mix. The processor is doing so much work renaming, reordering and predicting behind the scenes that second guessing and improving its work from C or assembly is between hard and nearly impossible. This realization is important in your optimization strategy, not a challenge to your skill—getting a 10x speedup from judicious penalty removal is highly unlikely on the Xbox One CPU. Some penalties do exist on this processor, but they are rare in typical code.

Caches & Memory Access Patterns More Important Than Ever

Although the L1 and L2 hardware mechanisms already try to prefetch data, they cannot compress or linearize data structures, and the predictor will work less well with very irregular access patterns. The same advice about L2 cache as on Xbox 360 applies:

Be careful with thread affinity and the split L2 cache. Two threads on different modules working with the same data set will incur heavy penalties through remote L2 and L1 latencies.

Work Harder to Avoid Branch Misprediction

The penalty for branch misprediction on an Xbox One core is 14 cycles. On a core able to retire 2 micro-operations per cycle, 14 cycles is a heavy penalty. Despite advanced branch prediction hardware, some constructs maximize the chance of a misprediction. Mispredictions cost more on Xbox One, relative to the Xbox 360, because it can maintain a higher IPC than the Xbox 360.

So, in very critical code, avoid virtual function calls to wide varieties of polymorphic classes from the same base class, function pointer dereferences, and switch statements based on effectively random data.

Multi-core Strategy is Critical

Xbox One has more hardware threads than Xbox 360, and each thread is far more able to support a heavy workload without affecting other threads. Multi-core Xbox 360 titles often rely on 2-3 heavy threads with light processing elsewhere, and the heavy threads often have relatively fixed functionality. In contrast, to get the very best performance out of the Xbox One CPU, it is crucial to find coherent ways to divide work and run many more tasks in parallel.

Preparation for Xbox One

There are several ways you can prepare for Xbox One from a CPU perspective:

Review your multi-core policy

Does your policy readily expand to more cores? Render submission is often bound by frame rate on Xbox 360 titles—is it possible to parallelize your render submission? You can try this with Direct3D 11 on Windows today with deferred contexts; console-specific optimizations are expected to give high performance in this area. Game logic is often a single, monolithic thread in Xbox 360 titles, which can be very difficult to disentangle into parallel pieces once written—is there a way to start pulling apart those pieces now?

Use 64-bit Addresses

Addresses in C/C++ on the Xbox One CPU are 64-bit. This will require some alteration to existing code unless you have been targeting 64-bit platforms already. Common issues are:

See References for best practices. In particular, the /wp64 compiler option produces 64-bit porting warnings on your 32-bit code (note that it can produce some false positives, especially on STL).

Convert Your SIMD and Intrinsics

If your engine contains SIMD code that does not use x86 instructions, you will need to convert it directly to the appropriate intrinsics. (Remembering to check whether you are using SIMD extensions supported by Xbox One.)

Alternatively, if you have VMX intrinsics from Xbox 360 code, another reasonable route is to convert those to use the DirectXMath library (formerly known as XNAMath), which will compile for targets that support SSE2. Converting VMX intrinsics to use DirectXMath will work on Xbox One cores, and it is already tested on x86-based and x64-based systems running Windows, and on Xbox 360.

Hand-Optimized Xbox 360 Code and the Peak Performance of Xbox One Cores

Overall, Xbox One cores will sustain a consistently higher IPC on each core than Xbox 360 cores can. However, in some rare cases, it is possible to hand-tune Xbox 360 code to avoid all penalties, maximize pipeline use, and completely avoid cache misses. This is difficult, but not impossible over short runs. In these cases, because the Xbox 360’s clock rate is double that of Xbox One’s, those sections of code might approach Xbox One’s sustained IPC. Processing that is designed around particular VMX instructions that have no direct equivalents in the SIMD extensions that Xbox One cores support may also prove difficult to match at half the clock rate.

References

Instruction Set Guides for x86/x64:

DirectXMath on MSDN Dev Center.

What’s New for Visual C++ in Visual Studio 11 Beta on MSDN, under “Visual C++ Compiler and Linker,” describes the auto-vectorizer available in Visual Studio 11 Preview.

Direct3D 11 Multithreading, a blog post on MSDN by Chuck Walbourn, links to resources about the multithreading features in Direct3D 11.

64-bit Gaming is Here Today, a presentation by Chuck Walbourn from Gamefest 2008, available from Download Center, about 64-bit programming; its references to Windows and older Visual Studio versions are not relevant to Xbox One.

64-bit Programming with Visual C++ on MSDN provides reference for setting up 64-bit builds on Windows, though some advice is not relevant to Xbox One.

Xbox One CPU Overview, a presentation from Xbox One Developer Summit 2012, available on Xbox Game Developer, that describes the instruction set, performance advantages, features of the architecture, how to prepare for migration, and the best focus for optimization efforts.