Working with NUI Speech recognition results

At the end of a speech recognition operation, the speech recognizer returns a result that contains information about the outcome of recognition. Apps typically use the recognition result to determine the user’s intent and what to do next, such as present a list of options, perform an action, confirm a user’s input, or prompt the user for additional input.

If recognition was successful (a user’s speech was matched to an active grammar), the recognition result includes the following information that applications typically find most useful:

• The text of the recognized phrase.
• A confidence rating assigned by the speech recognizer that indicates the certainty that the speech input matches the grammar.
• The semantic result defined in the SRGS grammar for the recognized phrase (only for SRGS grammars).

Recognized text

Recognized text is the phrase in the grammar that is the speech recognizer’s best match for speech input by a user. This may not be exactly what the user said, but it is the closest match to a phrase in an enabled grammar for what the user said. In grammars that do not contain semantics, an application typically uses the recognized text to interpret the user’s intent and to initiate an action in response.

Confidence rating

The confidence rating is the speech recognizer’s assessment of how accurately it matched a user’s speech to a phrase in an active grammar. A speech recognizer may assign a low confidence score to spoken input for various reasons, including background interference, inarticulate speech, or unanticipated words or word sequences.

The speech recognizer may return one or more possible recognized phrases (called alternates) in the result for a recognition operation. The alternates are phrases from the grammar. Recognition alternates may be used in any of the following ways:

• If there is only one alternate and its confidence score meets or exceeds the preset threshold, then the recognizer matches the alternate to the speech input.
• If there is only one alternate and its confidence score does not meet or exceed the preset threshold, then the alternate is rejected and the user’s speech is not recognized.
• If there are multiple recognition alternates for speech input that meet or exceed the preset threshold, and one alternate has a substantially higher confidence score than the other alternates, then the recognizer matches the higher scoring alternate to the speech input.

Semantic result

Semantic results (or simply “semantics”) are information that a grammar author enters when creating an XML-format grammar that conforms to the Speech Recognition Grammar Specification (SRGS) Version 1.0. Semantics assign meaning to the contents of an Item element or to a Ruleref element and the phrases they define. For example, a semantic assignment might assign an airport code to the name of each airport in a list. This way the airport code is returned, as well as the airport name, when a user speaks an airport name that matches the grammar. The following excerpt from an SRGS grammar illustrates how to assign a semantic value in the form of a string literal to the contents of an item element.

<rule id = "airports">
  <one-of>
    <item> Chicago <tag> "ORD" </tag> </item>
    <item> Boston <tag> "BOS" </tag> </item>
    <item> Miami <tag> "MIA" </tag> </item>
  </one-of>
</rule>  

A semantic assignment to a Ruleref element for Windows Phone could indicate that the airport name returned by one rule reference to a list of airports is the point of origin for a flight, while the airport name returned by another rule reference to the same list of airports is the flight’s destination. In the following example, the first rule reference to the rule named flightCities is directly followed by a Tag element. The tag element creates a property called LeavingFrom for the Rule Variable of the rule named flightBooker, and assigns the recognized text from the flightCities rule to the LeavingFrom property.

<rule id="flightBooker" scope="public">
  <item> I want to fly from </item>
  <ruleref uri="#flightCities" />
  <tag> out.LeavingFrom=rules.flightCities; </tag>
  <item> to </item>
  <ruleref uri="#flightCities" />
  <tag> out.GoingTo=rules.flightCities; </tag>
</rule>

<rule id="flightCities" scope="private">
  <one-of>
    <item> Chicago </item>
    <item> Boston </item>
    <item> Miami </item>
  </one-of>
</rule>