Media Foundation

The Media Foundation API supports applications and components for using digital media. Xbox One XDK implements a subset of the Microsoft Media Foundation API, supporting Encoding, Decoding, and Source Reader scenarios.

The Xbox One XDK XDK documentation includes a copy of the Reference topics based on the Media Foundation header files which are supported for Xbox One XDK; this is 7 out of 25 of the Media Foundation header files. For conceptual or overview topics, see Microsoft Media Foundation at MSDN.

Codecs and File Formats

Xbox One XDK supports the MP4 file format, the AAC audio codec, and the H.264 video codec.

The Xbox One XDK H.264 hardware video decoder has some requirements that are highly technical and will not apply in most applications, however if you are encoding your own content, be aware that some encoders have advanced options which enable the violation of some of the H.264 video codec specification. In the list below, if #1 is violated, decode will fail. If #2 or #3 are violated, the decoded video will show corruption in high motion frames. #4 is just informational.

1. Level 3.1 is the minimum level required to support 720p resolutions. As a consequence, 720p should not have sub-partitions, and H.264 compliance requires 16 motion vectors for video resolutions greater than 720p.
2. For levels above 3.1, the maximum number of motion vectors per two consecutive macroblocks is 16.
3. The MinLumaBiPredSize is 8x8. The maximum number of motion vectors for B frames is 8 and P frames is 16.
4. For resolutions less than 720p, up to 32 motion vectors is supported to allow sub-partition.

Media Foundation Transforms

The H.264 hardware encoder Media Foundation transform (MFT) on Xbox One is based on MFT for windows. On Xbox One, the encoder is an asynchronous MFT.

The Xbox One implementation of Media Foundation also supports the Source Reader for decoding MP4 files.

For encode, the only option is to follow the “Basic MFT Processing Model” and call ProcessInput/ProcessOutput on the encoder APIs in the processing loop. For decode, the game developer could follow the same model, or to make things easier use the Source Reader APIs. So there is only one way to use the encoder, while there are two ways to use the decoder. Developers would use the more manual ProcessInput/ProcessOutput method for decoding if they needed fine granularity of when decoding actually happened. SourceReader is easier to use, but does a lot of processing automatically with less control over when it happens.

The following topics on MSDN explain the relevant components of Media Foundation.

• Basic MFT Processing Model (Windows)
• Asynchronous MFT (Windows)
• Using the Source Reader to Process Media Data

APIs

To implement video encoding, decoding and rendering, refer to the following APIs:

• MFCreateDxvaSampleRendererX function
• MFCreateRGB32ToNV12ConverterX function
• MFCreateNV12ToRGB32ConverterX function
• IXboxColorConverter interface
• IXboxColorConverter::Transform method
• IXboxNV12MFSampleRenderer interface
• IXboxNV12MFSampleRenderer::RenderDecodedSample method

In this section

Media Foundation Transform Encoding
Describes how to implement H.264 hardware encoding support using the Media Foundation Transport (MFT) processing model on Xbox One Game OS.

Media Foundation Transform Decoding
Describes how to implement H.264 hardware decoding using the Source Reader in the Xbox One Game OS.

See also

Video API Reference