RFRProgress Machine Learning

The RFRProgress is an analog technology type. It uses the Random Forest Regression (RFR) algorithm to determine the progress of a gesture performed by a player. Also, it uses tagging models that are represented by analog signals that occur during a gesture.

Examples of these models could be an abstract concept such as the “progress” of a gesture or a quantifiable value such as the angle of an arm in a baseball pitch. Valid ranges for the RFRProgress tags are based on the model that the analog signal represents.

The input is an analog signal that models an action as progress during a gesture. For example, the position of the arm in a golf swing.

The RFRProgress is a context-based detector, meaning that the progress detection is only valid when a player is performing a gesture. The progress detection can be enabled as follows:

  1. When the title is aware that the player is performing a gesture, it enables the detection and uses it to aid in rendering a simulation.
  2. Progress detection is enabled when an AdaBoost trigger is active.

Note You can override the RFRProgress project settings in the solution’s global settings.

Input Parameters

The following table describes the input parameters that you can use when tagging a gesture. You enter these parameters in the Project Settings grid, as shown in the Visual Gesture Builder Training Project.

Note

Parameters marked (New) are new to the latest release. If you have an earlier VGB project, the project will be updated with the latest properties the first time when it is opened in the latest release, and you will be prompted to save the changes to the updated project.

Parameter Name Type Description
Use Rotation Variant Feature BOOL When set to true, the gesture detection will take into account the user’s global rotation. In this case the detection produces a stronger signal that is less robust to orientation change. When set to false, gesture detection will try to ignore global rotation as much as possible. For example golf swing with the user head on and golf swing with the user at 45 degrees returns a similar signal.
Number of Trees INT This parameter affects the accuracy, memory and disk space. Larger value gives better results at the cost of more memory and disk space, and also increases runtime CPU use for searching through these trees.
Maximum Tree Depth INT This parameter controls the maximal complexity of a decision tree. Larger value allows tree to grow deeper for complex scenarios, at the cost of more memory space and CPU use.
Cluster Threshold FLOAT The lower this value, the less change can occur from frame to frame. This could produce better overall results, however, there will be more misdetections.
Note
A smaller value means more localized search, which could be more accuracy if search is performed in the correct neighborhood. But it could also cause "stuck" in a neighborhood when no good matching could be found.
Weight Factor FLOAT The weight of the output from previous frame that will affect the current frame’s output. 0 means no affect at all.
Use Hands Data (New) BOOL Default to false. This means by default the hand states are not used for training and detection. For training and detection to use the hand states, set this property to true.
Caution
Hands data is only available for up to two players. If your game supports more than two simultaneous players, you should not use hands data during gesture training.
Ignore Left Arm (New) BOOL Default to false. This means by default the following left-arm joints are used for training.
  • NUI_SKELETON_JOINT_ELBOW_LEFT
  • NUI_SKELETON_JOINT_WRIST_LEFT
  • NUI_SKELETON_JOINT_HAND_LEFT
  • NUI_SKELETON_JOINT_HAND_TIP_LEFT
  • NUI_SKELETON_JOINT_THUMB_LEFT
For training to ignore the left-arm joints, set this property to true. This is useful when you train a right-hand gesture and when the signals from the left hand need to be ignored.
Ignore Right Arm (New)BOOLDefault to false. This means by default the following right-arm joints are used for training.
  • NUI_SKELETON_JOINT_ELBOW_RIGHT
  • NUI_SKELETON_JOINT_WRIST_RIGHT
  • NUI_SKELETON_JOINT_HAND_RIGHT
  • NUI_SKELETON_JOINT_HAND_TIP_RIGHT
  • NUI_SKELETON_JOINT_THUMB_RIGHT
For training to ignore the right-arm joints, set this property to true. This is useful when you train a left-hand gesture and when the signals from the right hand need to be ignored.
Ignore Lower Body (New)BOOLDefault to false. This means by default the following lower-body joints are used for training.
  • NUI_SKELETON_JOINT_KNEE_LEFT
  • NUI_SKELETON_JOINT_ANKLE_LEFT
  • NUI_SKELETON_JOINT_FOOT_LEFT
  • NUI_SKELETON_JOINT_KNEE_RIGHT
  • NUI_SKELETON_JOINT_ANKLE_RIGHT
  • NUI_SKELETON_JOINT_FOOT_RIGHT
For training to ignore the lower-body joints, set this property to true. This is useful when you train a gesture that uses upper body only and when you want a gesture to be applicable for both seated and standing positions.

Runtime Data

An RFRProgress gesture will be represented as a GestureType.Continuous Gesture during runtime. The ContinuousGestureResult Class should be used to check the progress value for the gesture.

Remarks

RFRProgress is a continuous gesture, which means that, unless it is disabled, it will always provide a progress value even when the player is not performing the gesture. The simplest way to know when the progress result is valid, is for your title to use a discrete gesture (AdaBoostTrigger) to help determine context.

For example, when the first frame is detected for the discrete gesture (Jump), enable the continuous gesture (JumpProgress). When the progress for the continuous gesture reaches a complete value, and/or the discrete gesture (Jump) is no longer detected, disable the continuous gesture (JumpProgress) so that it will no longer use resources while invalid.