Natural Language Understanding Annotation Set REST API Reference

Note: Sign in to the developer console to build or publish your skill.

Use the Natural Language Understanding Annotation (NLU) Annotation Set REST API to define annotation sets for your skill testing. An annotation set is a set of utterances mapped to expected intents and slots for your skill's interaction model. Each utterance with its expected intent and slots is called an annotation.

After you create an annotation set and add annotations, you can test the NLU accuracy of the interaction model. For more details, see NLU Evaluation REST API Reference.

If you want to create the annotation set in the Alexa developer console, see Batch Test Your NLU Model.

API endpoint

The endpoint of the Annotation Set API is https://api.amazonalexa.com.

Authentication

Each API request must have an authorization header whose value is the access token retrieved from Login with Amazon (LWA). For details, see Get an Access Token for SMAPI.

Operations

The Annotation Set API includes the following operations.

Operation	HTTP method and URI
Create annotation set	`POST /v1/skills/{skillId}/nluAnnotationSets`
Delete annotation set	`DELETE /v1/skills/{skillId}/nluAnnotationSets/{annotationId}`
Get annotation set contents	`GET /v1/skills/{skillId}/nluAnnotationSets/{annotationId}/annotations?maxResults={maxResults}&nextToken={nextToken}`
Get annotation set metadata	`GET /v1/skills/{skillId}/nluAnnotationSets/{annotationId}`
List annotation sets	`GET /v1/skills/{skillId}/nluAnnotationSets?nextToken={nextToken}&maxResults={maxResults}`
Update annotations	`POST /v1/skills/{skillId}/nluAnnotationSets/{annotationId}/annotations`

Create annotation set

Create a new annotation set. Add annotations to the set by calling Update annotations.

Request

To create an annotation set, you make a POST request to the nluAnnotationSets resource.

Request path and header example

Copied to clipboard.

POST /v1/skills/{skillId}/nluAnnotationSets
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}

Request path and header parameters

Parameter	Located in	Description	Type	Required
`skillId`	Path	Identifies the skill. Valid values: 1–255 characters.	String	Yes
`access token`	Header	LWA token.	String	Yes

Request body example

Copied to clipboard.

{
    "locale": "en-US",
    "name": "QuickTrivia annotation set1"
}

Request body properties

Property	Description	Type	Required
`locale`	Locale of the interaction model to test.	String	Yes
`name`	Name for the annotation set. Valid values: alphanumeric characters.	String	Yes

Response

A successful response returns HTTP 201 Created, along with a Location parameter in the response header. The response body contains the annotation set ID. Use the ID to reference and add sample utterances to the annotation set. On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.

Response body example

The following example shows a response.

{
    "id": "annotation.set.1"
}

Response body properties

Property	Description	Type
`id`	Identifies the new annotation set.	String

HTTP status codes

Status	Description
`201 Created`	Empty annotation set created successfully. The response header contains a `Location` parameter that includes the `skillId` and `annotationId`. The `annotationId` is the same as the `id` in the response body. For example: `HTTP 201 Created Content-Type: application/json Location: /v1/skills/{skillId}/nluAnnotationSets/{annotationId}/annotations`
`400 Bad Request`	Indicates that one or more properties in the request body aren't valid. The following example shows the response body with the error code and message. `{ "message": "The property is outside the allowed range.", "code": "INVALID_STRING_LENGTH" }`
`401 Unauthorized`	Request didn't include the authorization token, or the included token expired or isn't valid. Or, you don't have access to the resource.
`403 Forbidden`	Indicates that the authorization token is valid, but the requested operation isn't allowed.
`404 Not Found`	Requested resource not found.
`429 Too Many Requests`	Permitted rate limit, specified as number of requests per unit of time, exceeded. Retry the request by using exponential back-off.
`500 Server Error`	Error occurred on the server. Retry the request by using exponential back-off.
`503 Service Unavailable`	Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request.

Delete annotation set

Delete the specified NLU annotation set.

Request

To delete an annotation set, you make a DELETE request to the nluAnnotationSets resource.

Request path and header example

Copied to clipboard.

DELETE /v1/skills/{skillId}/nluAnnotationSets/{annotationId}
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json

Request path and header parameters

Parameter	Located in	Description	Type	Required
`skillId`	Path	Identifies the skill. Valid values: 1–255 characters.	String	Yes
`annotationId`	Path	Identifies the NLU annotation set to delete.	String	Yes
`access token`	Header	LWA token.	String	Yes

Request body example

The request has no body.

Request body properties

The request has no body.

Response

A successful response returns HTTP 204 No Content. On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.

Response body example

The response has no body.

Response body properties

The response has no body.

HTTP status codes

Status	Description
`204 No Content`	Annotation set deleted successfully.
`400 Bad Request`	Server can't process the request due to a client error. For example, the payload is malformed, the annotation set is malformed or empty. The response body contains a `violations` array with the errors. The following example shows the response body with a validation error. `{ "message": "Payload validations failed", "violations": [{ "message": "The \"name\" property at property path \"$.name\" is outside the allowed range.", "code": "INVALID_STRING_LENGTH" }] }`
`401 Unauthorized`	Authorization token is invalid, expired, or doesn't have access to the resource.
`403 Forbidden`	The requested operation isn't allowed. The response body contains a `violations` array with the errors.
`404 Not Found`	The requested resource isn't found. For example, the `skillId` isn't found. The following example shows the response body with the error and message. `{ "message": "The skill cannot be found.", "code": "RESOURCE_NOT_FOUND" }`
`409 Conflict`	Request to run an evaluation for a given skill conflicts with an evaluation request that's currently in progress.
`429 Too Many Requests`	User has exceeded the permitted rate limit.
`500 Internal Server Error`	Server has encountered an error.
`503 Service Unavailable`	Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request.

Get annotation set contents

Download the contents of the specified NLU annotation set in comma-separated values (CSV) or JSON format.

Request

To download the contents of the annotation set, you make a GET request to the nluAnnotationSets resource.

Request path and header example

Copied to clipboard.

GET /v1/skills/{skillId}/nluAnnotationSets/{annotationId}/annotations?maxResults={maxResults}&nextToken={nextToken}
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: {accept-type}

Request path and header parameters

Parameter	Located in	Description	Type	Required
`skillId`	Path	Identifies the skill. Valid values: 1–255 characters.	String	Yes
`annotationId`	Path	Identifies the annotation set.	String	Yes
`maxResults`	Query	Maximum number of results to return in the response. Include when the accept header indicates `application/json` format only. Default: 1000.	Number	No
`nextToken`	Query	Token from the previous response. Include if iterating over a paginated response. Include when the accept header indicates `application/json` format only.	String	No
`access token`	Header	LWA token.	String	Yes
`accept-type`	Header	Specifies the requested format of the returned contents. Valid values: `text/csv`, `application/json`.	String	Yes

Request body example

The request has no body.

Request body properties

The request has no body.

Response

A successful response returns HTTP 200 OK, along with the contents of the annotation set in the specified format.

On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.

CSV response body example

When you specify CSV format, the response body includes the entire contents. The first row of the response body contains comma-delimited attribute names of the annotation properties. Subsequent rows define the annotations. The annotation properties in each row follow the strict ordering of the attribute property names defined in the first row.

The following example shows a response in CSV format.

utterance, referenceTimestamp, intent, slot[toCity], slot[fromCity], slot[travelDate]

plan a trip,,PlanMyTripIntent,,,

i want to go to chicago on monday,2019-08-29T00:00:00.000Z,PlanMyTripIntent,chicago,,2019-08-30

plan a trip from seattle to boston,,PlanMyTripIntent,boston,seattle,

plan a trip from chicago to denver on Monday,2019-08-29T00:00:00.000Z,PlanMyTripIntent,denver,chicago,2019-09-02

CSV response body properties

The order of the columns doesn't matter because the first line of the CSV payload specifies the column ordering.

Property	Description	Type
`utterance`	Utterance to test. Don't include the wake word or invocation name. You can use either written form or spoken form for the utterance. For example, you can use numerals ("5") or write out numbers ("five"). For more examples, see the rules for custom slot type values.	String
`referenceTimestamp`	Date and time to use as the basis for relative date and time values. Include when the utterance tests. Use this when the utterance tests `AMAZON.DATE` or `AMAZON.TIME` slots with words that represent relative dates and times such as "today," "tomorrow," and "now." Defined in ISO 8601 format, `YYYY-MM-DDThh:mm:ssZ`. For more details about testing relative date and time values, see Create an annotation with relative dates or times.	String
`intent`	Name of the expected intent for the utterance.	String
`slot`	List of expected slots for the utterance.	Array of objects
`slot[slotname]`	Expected value for the slot.	String
`slots[slotname][index]`	Expected values for the slot. For a slot that accepts multiple values, add an index number in brackets. To account for a single value for this slot, add the slot without an index.	String

JSON response body example

The following example shows a response in JSON format.

{
  "data": [
    {
      "inputs": {
        "utterance": "plan a trip"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent"
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "i want to go to chicago on monday",
        "referenceTimestamp": "2020-12-11T12:00:00.000Z"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Chicago",
                  "type": "Simple"
                }
              },
              "travelDate": {
                "slotValue": {
                  "value": "2020-12-14",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip from seattle to boston"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Boston",
                  "type": "Simple"
                }
              },
              "fromCity": {
                "slotValue": {
                  "value": "Seattle",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip from chicago to denver on thursday",
        "referenceTimestamp": "2020-12-11T12:00:00.000Z"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Denver",
                  "type": "Simple"
                }
              },
              "fromCity": {
                "slotValue": {
                  "value": "Chicago",
                  "type": "Simple"
                }
              },
              "travelDate": {
                "slotValue": {
                  "value": "2020-12-17",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip to missoula to go camping, hiking, and fishing"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Missoula",
                  "type": "Simple"
                }
              },
              "activity": {
                "slotValue": {
                  "type": "List",
                  "values": [
                    {
                      "value": "camping",
                      "type": "Simple"
                    },
                    {
                      "value": "hiking",
                      "type": "Simple"
                    },
                    {
                      "value": "fishing",
                      "type": "Simple"
                    }
                  ]
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip to portland to go kayaking"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Portland",
                  "type": "Simple"
                }
              },
              "activity": {
                "slotValue": {
                  "value": "kayaking",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    }
  ]
}

JSON response body properties

Property	Description	Type
`inputs`	Input to the NLU evaluation.	Object
`inputs.utterance`	Utterance to test. Don't include the wake word or invocation name. You can use either written form or spoken form for the utterance. For example, you can use numerals ("5") or write out numbers ("five"). For more examples, see the rules for custom slot type values.	String
`inputs.referenceTimestamp`	Date and time to use as the basis for relative date and time values. Include when the utterance tests. Use this when the utterance tests `AMAZON.DATE` or `AMAZON.TIME` slots with words that represent relative dates and times such as "today," "tomorrow," and "now." Defined in ISO 8601 format, `YYYY-MM-DDThh:mm:ssZ`. For more details about testing relative date and time values, see Create an annotation with relative dates or times.	String
`expected`	Expected results.	Object
`expected.intent`	Expected intent and slot values.	Object
`expected.intent.name`	Name of the expected intent for the utterance.	String
`expected.intent.slots`	List of expected slots for the utterance.	Array of objects
`expected.intent.slots.<slotname>.slotValue`	Expected value for the slot. The `slotValue` object has either a `value` or `values` property, but never both.	Object
`expected.intent.slots.<slotname>.slotValue.type`	Slot type. Valid values: For a single value slot, set to `Simple` and include a `value` property with the string value of the slot. For a multiple-value slot, set to `List` and include a `values` property with a list of slot values.	String
`expected.intent.slots.<slotname>.slotValue.value`	Include for a single-value slot only. Contains the expected value of the slot. Omit for a multiple-value slot.	String
`expected.intent.slots.<slotname>.slotValue.values`	Include for a multiple-value slot only. Contains a list of expected values for the slot. Each object in the array has a value property for the string value and a type property. Specify the values in the same order as they're written in the utterance. Omit for a single-value slot.	Array of objects
`expected.intent.slots.<slotname>.slotValue.values[].type`	Slot type. Valid value: `Simple`.	String
`expected.intent.slots.<slotname>.slotValue.values[].value`	Contains the expected value of the slot.	String

HTTP status codes

Status	Description
`200 OK`	Response body contains the contents of the NLU annotation set.
`400 Bad Request`	Server can't process the request due to a client error. For example, the payload is malformed, the annotation set is malformed or empty. The response body contains a `violations` array with the errors. The following example shows the response body with a validation error. `{ "message": "Payload validations failed", "violations": [{ "message": "The \"name\" property at property path \"$.name\" is outside the allowed range.", "code": "INVALID_STRING_LENGTH" }] }`
`401 Unauthorized`	Authorization token is invalid, expired, or doesn't have access to the resource.
`403 Forbidden`	The requested operation isn't allowed. The response body contains a `violations` array with the errors.
`404 Not Found`	The requested resource isn't found. For example, the `skillId` isn't found. The following example shows the response body with the error and message. `{ "message": "The skill cannot be found.", "code": "RESOURCE_NOT_FOUND" }`
`409 Conflict`	Request to run an evaluation for a given skill conflicts with an evaluation request that's currently in progress.
`429 Too Many Requests`	User has exceeded the permitted rate limit.
`500 Internal Server Error`	Server has encountered an error.
`503 Service Unavailable`	Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request.

Get annotation set metadata

Get the metadata for the specified annotation set.

Request

To get the metadata for an annotation set, you make a GET request to the nluAnnotationSets resource.

Request path and header example

Copied to clipboard.

GET /v1/skills/{skillId}/nluAnnotationSets/{annotationId}
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json

Request path and header parameters

Parameter	Located in	Description	Type	Required
`skillId`	Path	Identifies the skill. Valid values: 1 - 255 characters.	String	Yes
`annotationId`	Path	Identifies the annotation set.	String	Yes
`access token`	Header	LWA token.	String	Yes

Request body example

The request has no body.

Request body properties

The request has no body.

Response

A successful response returns HTTP 200 OK, along with the annotation set metadata. On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.

Response body example

{
    "locale": "en-US",
    "name": "QuickTrivia annotation set1",
    "numberOfEntries": 12,
    "updatedTimestamp": "2019-08-12T18:46:31.674Z"
}

Response body properties

Property	Description	Type
`locale`	Locale of the annotation set.	String
`name`	Name of the annotation set.	String
`numberOfEntries`	Number of annotations in the annotation set.	Integer
`updatedTimestamp`	Timestamp of the last update to the annotation set. Defined in ISO 8601 format, `YYYY-MM-DDThh:mm:ssZ`.	String

HTTP status codes

Status	Description
`200 OK`	Response body contains the metadata for the NLU annotation set.
`400 Bad Request`	Server can't process the request due to a client error. For example, the payload is malformed, the annotation set is malformed or empty. The response body contains a `violations` array with the errors. The following example shows the response body with a validation error. `{ "message": "Payload validations failed", "violations": [{ "message": "The \"name\" property at property path \"$.name\" is outside the allowed range.", "code": "INVALID_STRING_LENGTH" }] }`
`401 Unauthorized`	Authorization token is invalid, expired, or doesn't have access to the resource.
`403 Forbidden`	The requested operation isn't allowed. The response body contains a `violations` array with the errors.
`404 Not Found`	The requested resource isn't found. For example, the `skillId` isn't found. The following example shows the response body with the error and message. `{ "message": "The skill cannot be found.", "code": "RESOURCE_NOT_FOUND" }`
`409 Conflict`	Request to run an evaluation for a given skill conflicts with an evaluation request that's currently in progress.
`429 Too Many Requests`	User has exceeded the permitted rate limit.
`500 Internal Server Error`	Server has encountered an error.
`503 Service Unavailable`	Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request.

List annotation sets

Get all NLU annotation sets for the specified skill.

Request

To list all annotation sets for the specified skill, you make a GET request to the nluAnnotationSets resource.

Request path and header example

Copied to clipboard.

GET /v1/skills/{skillId}/nluAnnotationSets?nextToken={nextToken}&maxResults={maxResults}
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json

Request path and header parameters

Parameter	Located in	Description	Type	Required
`skillId`	Path	Identifies the skill. Valid values: 1–255 characters.	String	Yes
`maxResults`	Query	Maximum number of results to return in the response. Default: 1000.	Number	No
`nextToken`	Query	Token from the previous response. Include if iterating over a paginated response.	String	No
`access token`	Header	LWA token.	String	Yes

Request body example

The request has no body.

Request body properties

The request has no body.

Response

A successful response returns HTTP 200 OK, along with a list of annotation sets. On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.

Response body example

{
    "annotationSets": [{
        "locale": "en-US",
        "name": "QuickTrivia annotation set1",
        "numberOfEntries": 1,
        "updatedTimestamp": "2019-08-12T18:46:31.674Z",
        "annotationId": "annotation.set.1"
    }],
    "paginationContext": {
        "nextToken": "someToken.1"
    },
    "_links": {
        "self": {
            "href": "/v1/skills/amzn1.ask.skill.1/nluAnnotationSets"
        },
        "next": {
            "href": "/v1/skills/amzn1.ask.skill.1/nluAnnotationSets?nextToken=someToken.1&maxResults=100"
        }
    }
}

Response body properties

Property	Description	Type
`annotationSets`	List of annotation sets.	Array of objects
`annotationSets[].locale`	Locale of the annotation set.	String
`annotationSets[].name`	Name of the annotation set.	String
`annotationSets[].numberOfEntries`	Number of utterances in the annotation set.	Integer
`annotationSets[].updatedTimestamp`	Timestamp of the last update to the annotation set. Defined in ISO 8601 format, `YYYY-MM-DDThh:mm:ssZ`.	String
`annotationSets[].annotationId`	Identifies the NLU annotation set.	String
`paginationContext`	Indicates there are more matching results. If present, the response contains a subset of the results. When there are no more matching results, `paginationContext` isn't included in the response.	Object
`paginationContext.nextToken`	Identifies the next set of annotations to return. The token expires in 24 hours. Use this value for the `nextToken` parameter in a subsequent List annotation sets request.	String
`_links`	Links for result navigation. Defined in JSON Hypertext Application Language specification.	_links object

HTTP status codes

Status	Description
`200 OK`	Response body contains a list of NLU annotation sets.
`400 Bad Request`	Server can't process the request due to a client error. For example, the payload is malformed, the annotation set is malformed or empty. The response body contains a `violations` array with the errors. The following example shows the response body with a validation error. `{ "message": "Payload validations failed", "violations": [{ "message": "The \"name\" property at property path \"$.name\" is outside the allowed range.", "code": "INVALID_STRING_LENGTH" }] }`
`401 Unauthorized`	Authorization token is invalid, expired, or doesn't have access to the resource.
`403 Forbidden`	The requested operation isn't allowed. The response body contains a `violations` array with the errors.
`404 Not Found`	The requested resource isn't found. For example, the `skillId` isn't found. The following example shows the response body with the error and message. `{ "message": "The skill cannot be found.", "code": "RESOURCE_NOT_FOUND" }`
`409 Conflict`	Request to run an evaluation for a given skill conflicts with an evaluation request that's currently in progress.
`429 Too Many Requests`	User has exceeded the permitted rate limit.
`500 Internal Server Error`	Server has encountered an error.
`503 Service Unavailable`	Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request.

Update annotations

Add or update annotations in an existing annotation set. Before you add or update annotations, make sure that you create an annotation set.

Note: The maximum number of annotations per annotation set is 10,000.

Request

To update the annotation set, you make a POST request to the nluAnnotationSets resource.

Request path and header example

Copied to clipboard.

POST /v1/skills/{skillId}/nluAnnotationSets/{annotationId}/annotations
Host: api.amazonalexa.com
Content-Type: {content-type}
Authorization: Bearer {access token}
Accept: application/json

Request path and header parameters

Parameter	Located in	Description	Type	Required
`skillId`	Path	Identifies the skill. Valid values: 1–255 characters.	String	Yes
`annotationId`	Path	Identifies the annotation set.	String	Yes
`content-type`	Header	Specifies the format of the included updates. Valid values: `text/csv`, `application/json`.	String	Yes
`access token`	Header	LWA token.	String	Yes

CSV request body example

When you specify CSV format, the request body includes the entire contents. The first row of the request body must contain comma-delimited attribute names of the annotation properties. Subsequent rows define the annotations. The annotation properties in each row must follow the strict ordering of the attribute property names defined in the first row. For multi-value slot names, duplicate the slot[SlotName] and add an index number in brackets. Include the slot[SlotName] at the end to use when the utterance includes a single value for the multi-value slot.

The following example shows a request in UTF8-encoded CSV format.

utterance, referenceTimestamp, intent, slot[toCity],slot[fromCity],slot[travelDate],slot[activity][0],slot[activity][1],slot[activity][2],slot[activity]

plan a trip,,PlanMyTripIntent,,,

i want to go to chicago on monday,2019-08-29T00:00:00.000Z,PlanMyTripIntent,chicago,,2019-08-30

plan a trip from seattle to boston,,PlanMyTripIntent,boston,seattle,

plan a trip from chicago to denver on Monday,2019-08-29T00:00:00.000Z,PlanMyTripIntent,denver,chicago,2019-09-02

plan a trip to missoula to go camping hiking and fishing,,PlanMyTripIntent,Missoula,,,camping,hiking,fishing

plan a trip to portland to go kayaking,,PlanMyTripIntent,Portland,,,,,,kayaking

CSV request body properties

The order of the columns doesn't matter because the first line of the CSV payload specifies the column ordering.

Property	Description	Type	Required
`utterance`	Utterance to test. Don't include the wake word or invocation name. You can use either written form or spoken form for the utterance. For example, you can use numerals ("5") or write out numbers ("five"). For more examples, see the rules for custom slot type values.	String	Yes
`referenceTimestamp`	Date and time to use as the basis for relative date and time values. Include when the utterance tests. Use this when the utterance tests `AMAZON.DATE` or `AMAZON.TIME` slots with words that represent relative dates and times such as "today," "tomorrow," and "now." Defined in ISO 8601 format, `YYYY-MM-DDThh:mm:ssZ`. For more details about testing relative date and time values, see Create an annotation with relative dates or times.	String	No
`intent`	Name of the expected intent for the utterance.	String	Yes
`slot`	List of expected slots for the utterance.	Array of objects	No
`slot[slotname]`	Expected value for the slot.	String	No
`slots[slotname][index]`	For a slot that accepts multiple values, add an index number in brackets. To account for a single value for this slot, add the slot without an index.	String	No

JSON request body example

The following example shows a request in JSON format.

{
  "data": [
    {
      "inputs": {
        "utterance": "plan a trip"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent"
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "i want to go to chicago on monday",
        "referenceTimestamp": "2020-12-11T12:00:00.000Z"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Chicago",
                  "type": "Simple"
                }
              },
              "travelDate": {
                "slotValue": {
                  "value": "2020-12-14",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip from seattle to boston"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Boston",
                  "type": "Simple"
                }
              },
              "fromCity": {
                "slotValue": {
                  "value": "Seattle",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip from chicago to denver on thursday",
        "referenceTimestamp": "2020-12-11T12:00:00.000Z"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Denver",
                  "type": "Simple"
                }
              },
              "fromCity": {
                "slotValue": {
                  "value": "Chicago",
                  "type": "Simple"
                }
              },
              "travelDate": {
                "slotValue": {
                  "value": "2020-12-17",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip to missoula to go camping, hiking, and fishing"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Missoula",
                  "type": "Simple"
                }
              },
              "activity": {
                "slotValue": {
                  "type": "List",
                  "values": [
                    {
                      "value": "camping",
                      "type": "Simple"
                    },
                    {
                      "value": "hiking",
                      "type": "Simple"
                    },
                    {
                      "value": "fishing",
                      "type": "Simple"
                    }
                  ]
                }
              }
            }
          }
        }
      ]
    },
    {
      "inputs": {
        "utterance": "plan a trip to portland to go kayaking"
      },
      "expected": [
        {
          "intent": {
            "name": "PlanMyTripIntent",
            "slots": {
              "toCity": {
                "slotValue": {
                  "value": "Portland",
                  "type": "Simple"
                }
              },
              "activity": {
                "slotValue": {
                  "value": "kayaking",
                  "type": "Simple"
                }
              }
            }
          }
        }
      ]
    }
  ]
}

JSON request body properties

Property	Description	Type	Required
`inputs`	Input to the NLU evaluation.	Object	Yes
`inputs.utterance`	Utterance to test. Don't include the wake word or invocation name. You can use either written form or spoken form for the utterance. For example, you can use numerals ("5") or write out numbers ("five"). For more examples, see the rules for custom slot type values.	String	Yes
`inputs.referenceTimestamp`	Date and time to use as the basis for relative date and time values. Include when the utterance tests. Use this when the utterance tests `AMAZON.DATE` or `AMAZON.TIME` slots with words that represent relative dates and times such as "today," "tomorrow," and "now." Defined in ISO 8601 format, `YYYY-MM-DDThh:mm:ssZ`. For more details about testing relative date and time values, see Create an annotation with relative dates or times.	String	No
`expected`	Expected results.	Object	Yes
`expected.intent`	Expected intent and slot values.	Object	Yes
`expected.intent.name`	Name of the expected intent for the utterance.	String	Yes
`expected.intent.slots`	List of expected slots for the utterance.	Array of objects	No
`expected.intent.slots.<slotname>.slotValue`	Expected value for the slot. The `slotValue` object has either a `value` or `values` property, but never both.	Object	No
`expected.intent.slots.<slotname>.slotValue.type`	Slot type. Valid values: For a single value slot, set to `Simple` and include a `value` property with the string value of the slot. For a multiple-value slot, set to `List` and include a `values` property with a list of slot values.	String	Yes
`expected.intent.slots.<slotname>.slotValue.value`	Include for a single-value slot only. Contains the expected value of the slot. Omit for a multiple-value slot.	String	No
`expected.intent.slots.<slotname>.slotValue.values`	Include for a multiple-value slot only. Contains a list of expected values for the slot. Each object in the array has a value property for the string value and a type property. Specify the values in the same order as they're written in the utterance. Omit for a single-value slot.	Array of objects	No
`expected.intent.slots.<slotname>.slotValue.values[].type`	Slot type. Valid value: `Simple`.	String	Yes
`expected.intent.slots.<slotname>.slotValue.values[].value`	Contains the expected value of the slot.	String	Yes

Response

A successful response returns HTTP 204 No Content. On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.

Response body example

The response has no body.

Response body properties

The response has no body.

HTTP status codes

Status	Description
`HTTP 204 No Content`	Annotation set updated successfully.
`400 Bad Request`	Server can't process the request due to a client error. For example, the payload is malformed, the annotation set is malformed or empty. The response body contains a `violations` array with the errors. The following example shows the response body with a validation error. `{ "message": "Payload validations failed", "violations": [{ "message": "The \"name\" property at property path \"$.name\" is outside the allowed range.", "code": "INVALID_STRING_LENGTH" }] }`
`401 Unauthorized`	Authorization token is invalid, expired, or doesn't have access to the resource.
`403 Forbidden`	The requested operation isn't allowed. The response body contains a `violations` array with the errors.
`404 Not Found`	The requested resource isn't found. For example, the `skillId` isn't found. The following example shows the response body with the error and message. `{ "message": "The skill cannot be found.", "code": "RESOURCE_NOT_FOUND" }`
`409 Conflict`	Request to run an evaluation for a given skill conflicts with an evaluation request that's currently in progress.
`429 Too Many Requests`	User has exceeded the permitted rate limit.
`500 Internal Server Error`	Server has encountered an error.
`503 Service Unavailable`	Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request.

Update metadata

Update the metadata for the specified annotation set.

Request

To update the annotation set, you make a PUT request to the nluAnnotationSets resource.

Request path and header example

Copied to clipboard.

PUT /v1/skills/{skillId}/nluAnnotationSets/{annotationId}
Host: api.amazonalexa.com
Content-Type: application/json
Authorization: Bearer {access token}
Accept: application/json

Request path and header parameters

Parameter	Located in	Description	Type	Required
`skillId`	Path	Identifies the skill. Valid values: 1–255 characters.	String	Yes
`annotationId`	Path	Identifies the NLU annotation set.	String	Yes
`access token`	Header	LWA token.	String	Yes

Request body example

Copied to clipboard.

{
    "name": "QuickTrivia annotation set10"
}

Request body properties

Property	Description	Type	Required
`name`	Name for the NLU annotation set. Valid value: alphanumeric characters.	String	Yes

Response

A successful response returns HTTP 201 Created. On error, the response returns the appropriate HTTP status code and includes a response body with an error code and human readable message.

Response body example

The response has no body.

Response body properties

The response has no body.

HTTP status codes

Status	Description
`HTTP 201 Created`	Annotation set properties updated successfully.
`400 Bad Request`	Server can't process the request due to a client error. For example, the payload is malformed, the annotation set is malformed or empty. The response body contains a `violations` array with the errors. The following example shows the response body with a validation error. `{ "message": "Payload validations failed", "violations": [{ "message": "The \"name\" property at property path \"$.name\" is outside the allowed range.", "code": "INVALID_STRING_LENGTH" }] }`
`401 Unauthorized`	Authorization token is invalid, expired, or doesn't have access to the resource.
`403 Forbidden`	The requested operation isn't allowed. The response body contains a `violations` array with the errors.
`404 Not Found`	The requested resource isn't found. For example, the `skillId` isn't found. The following example shows the response body with the error and message. `{ "message": "The skill cannot be found.", "code": "RESOURCE_NOT_FOUND" }`
`409 Conflict`	Request to run an evaluation for a given skill conflicts with an evaluation request that's currently in progress.
`429 Too Many Requests`	User has exceeded the permitted rate limit.
`500 Internal Server Error`	Server has encountered an error.
`503 Service Unavailable`	Server is down for maintenance, overloaded, or otherwise unavailable to handle the incoming request.

Object definitions

The NLU Annotation Set API defines the following objects.

Locale values

The following table shows valid values for the locale property.

Locale code	Language
`ar-SA`	Arabic (SA)
`de-DE`	German (DE)
`en-AU`	English (AU)
`en-CA`	English (CA)
`en-GB`	English (UK)
`en-IN`	English (IN)
`en-US`	English (US)
`es-ES`	Spanish (ES)
`es-MX`	Spanish (MX)
`es-US`	Spanish (US)
`fr-CA`	French (CA)
`fr-FR`	French (FR)
`hi-IN`	Hindi (IN)
`it-IT`	Italian (IT)
`ja-JP`	Japanese (JP)
`nl-NL`	Dutch (NL)
`pt-BR`	Portuguese (BR)

Utterance Profiler REST API Reference

Was this page helpful?

Provide feedback

Last updated: Aug 01, 2024