Pre-Release Titles: Testing Your Xbox One Product with a Beta

This topic provides an overview of operating a beta test on an Xbox One title before it is released as a final product. The goal of the document is to familiarize Xbox One game developers and producers with how a beta can be used on Xbox One and to provide information regarding the design choices available for betas.

This document covers recommended paths and alternative solutions but might not meet the needs of all products. If there is a unique scenario you would like to see supported, please work with your account team.

An Xbox One beta: what is it?

To ensure that users get the stable and enjoyable experiences that they expect on their Xbox One consoles, title developers, producers, and other partners need to be able to test their products on a large scale before a complete title release. To allow partners the flexibility to learn at scale while still protecting users, partners can release a beta of their titles over Xbox Live. In this context, the term beta is not used in relation to development milestones (alpha, beta, etc.). Instead, it refers to a non-final release of a title that is provided to a subset of users on Xbox Live. Partners can use different milestones in managing their own development schedules, but a beta on Xbox One is a non-final version of software that is released to users.

At this time, a beta can be released only as a stand-alone product and is updated through a standard content update. There is no current feature set that allows beta groups, selective updates, or A/B testing on a released product. Betas can be released to selected markets or to a selected audience through tokens, but any differentiated group testing needs to be done using separate beta products.

Selecting your beta audiences

When you release a beta on Xbox One, you must first choose the audience to whom you want to give access to the content. This choice will directly affect your design choices for features, tools, distribution methods, and certification throughout the beta program. After the audience type has been chosen, you can expand the audience but not constrict it-you can add external users to an internal beta (after notifying Microsoft and passing certification), but you cannot separate those audiences later. Because of this restriction, you might prefer to run two betas concurrently, one for each audience.

Internal beta release

An internal beta is one that is released only to individuals who either are employees of the title publisher or its subsidiary companies or are employees of any company the title publisher partnered with in developing the beta. Internal beta releases are limited to a maximum of 10,000 participants, and distribution of the beta must be completed using Microsoft-provided tokens.

Because they are released only to employees or subsidiaries of the partner, internal betas do not require a Microsoft certification pass. However, they are restricted in the feature sets that are available to them because of the inherent risk involved in releasing untested products. Internal betas allow the easiest test path for getting quick information about a title, but to minimize potential risks to users, the scope of feature sets available in internal betas is smaller than the scope of either external betas or full title releases. See Feature limitations and security in this document for details.

External/public beta

An external or public beta is one that can be distributed, at the partner’s discretion, to the public, friends and family, and the media. There is no limit to the audience size of a public beta, and a partner can choose to deliver the beta through a free offer or using Microsoft-provided tokens.

Most of the features on Xbox Live are available to external betas, and the scalability of external beta releases provides partners with more thorough test information. Users do have limited expectations about the quality of beta releases, so there is a reduced certification path for external Xbox One betas to cover the most basic requirements of the platform. For more information, see Certification and Xbox Requirements, below.

Setting up the beta product type

After you have selected the audience or audiences for your beta, you must determine how you want to roll out the beta to your audience. If you are planning a two-part beta-internal and external-you might use the same TitleID for both the internal and external portions. Alternatively, you might prefer to use different TitleIDs to keep the audiences separate. Both options are supported and are at the partner’s discretion. If your plan is to run your beta in separate stages, be aware that migrating between audiences involves certain implications that must be considered; see How to extend your beta audience, below, for more information.

DGameDemo

This is the required product type for games on Xbox One. By using a demo product type, you will use a different TitleID than your final product, and you won’t affect areas such as instrumentation, achievements, and purchases in that final product. Because your beta will remain a part of the same product group, you’ll be able to avoid additional management challenges while maintaining the clear separation between the beta product and final product.

Using a demo product type for your beta offers several benefits:

There are also certain challenges associated with using a demo product type for your beta. For example:

Feature limitations and security

Because a beta must be configured as a DGameDemo, there are limitations on the feature set available compared to that of a full game. The following table outlines the consumer services that are available for demos versus full titles.

Feature DGameDemo DGame Notes
Presence Available Available Recommendations:
  • Internal: Restrict
  • External: Expose
Achievements Not available Available  
Instrumentation / Events Available Available Instrumentation collected in the DGameDemo is separate from the instrumentation collected in the full DGame product.
Hero Stats Available Available  
Game-created GameDVR clips Not available Available  
“Xbox record that” GameDVR clips Available Available Can be restricted based on partner requests to maintain confidentiality.

Maintaining security

When you are releasing a beta, confidentiality and security are important considerations. Although you cannot control whether people take pictures of their screens with a camera, you can perform several activities to maintain the security of your IP or beta programs:

Beta submission

Betas that you submit to Microsoft are subject to the same process and minimum requirements, based on feature sets chosen, as any other standard Xbox One submission.

In general, titles must include:

  1. Submission validator logs.
  2. Content metadata and screen shots.
    • All marketing strings. Code names can be used for internal betas.
    • Genre.
    • Rating—can be unrated or rating pending if the rating has not been granted yet.
    • All artwork other than screenshots. Placeholders can be used for hidden offers.

      Note In visible betas, this information/content must adhere to the same standards as full titles, including the Xbox Store localization policy.

  3. Service configuration for instrumentation and related items.
  4. Rating certificates, if available. Our expectation is that most titles will carry a rating of RP in the United States.

To help simplify your submission, please work with your account manager to determine what resources are available for you to use.

Certification and Xbox Requirements

All titles, including beta titles, must enter and pass certification before being released into the retail environment. However, the Xbox Requirements (XRs) that will be applicable and the depth of the certification process will vary depending on the audience the content will be exposed to.

Internal betas

As noted above, internal betas may be exposed only to the staff of the publisher or its subsidiaries or development partners. For this reason, the only certification testing performed on internal betas will be the Submission Validator portions of XR-003, Title Quality for Submission.

External betas

Because external betas are released to the full Xbox One user community, the certification pass required for the beta to be released into retail is greater than that required for internal betas. It is less than that required for full title retail release, but it also includes additional requirements that apply specifically to betas.

Beta-specific Xbox Requirements:

In addition to the beta-specific requirements outlined above, all external betas must meet the following XRs for certification. The XRs that are denoted with a * undergo certification testing.

Delivering your beta to users

Betas are released in the retail environment, so the content must be signed and distributed through Xbox Live. The beta could theoretically be delivered on disc, but this scenario is not currently supported.

Beta distribution can take place only through free or hidden offers; they cannot require payment to access. The availability of such offers will be determined largely by the audience to which the partner will be exposing the content-internal versus external-and whether the distribution will be further limited, such as by size or region or as a “bonus” offer.

Tokens

We expect that the most common distribution method for betas will be through token-only offers. With this method, partners order tokens for their betas and provide them to their audience as they see fit, other than directly selling acces). This method is the only means of distribution available to internal betas. The token-ordering process is no different from any other Xbox Live token orders; please work with your account manager if you need more information.

Hidden offers

Partners releasing an external beta can expose the beta as a hidden offer that can be accessed from within another experience, such as another game title or hub application. This provides a simple distribution method expecially for connected title experiences. This method does not allow for strict control of content in most cases.

Public offers

Last, and simplest, titles can make an external beta available as a public offer on Xbox Live. In this scenario, any and all users of Xbox Live could download and access the beta.

Beta updates

The process for updating beta releases is the same as the standard process for all content updates on Xbox One. When a title is updated, the update is downloaded to users based on their current system settings, and any new users of the title will download the most current, updated version.

Certification for a beta update remains the same as the original submission of the beta, including the associated requirements and test cases.

How to extend your beta audience

If you have chosen to use a single TitleID for both an internal and external beta, you must notify Microsoft before providing beta access to the external audience. This will allow Microsoft to complete the required certification pass against the reduced XRs for beta titles. And because an internal beta is already in the retail environment, testing for the transition to an external beta will also usually occur in that environment.

If all applicable Xbox Requirements are passed successfully and there have been no changes to artwork and so forth, you can begin distributing the beta offer externally without making any changes to the original internal beta.

Ending your beta

As your full product release approaches, you will want to end your beta experience for existing users. Partners are responsible for determining their beta end game, and there are many ways a partner can gracefully exit a beta experience. For example:

In addition, no matter how you choose to restrict access to the beta, you must also expire the offer in the Xbox Store, nullifying any existing tokens and preventing future downloads.