Cloudside Integration Overview (VSK Fire TV)

To implement the VSK for your Fire TV app, you primarily implement the Alexa Video Skill API. However, with cloudside integrations, you will also involve additional APIs and services, including Amazon Device Messaging (ADM), AWS Lambda, AWS IAM, Login with Amazon, Node JS, Alexa Client Library, and more. Incorporating the VSK for your Fire TV app gives customers the richest voice experience with your app, driving up the levels of engagement and discovery for your content.

• Overview
• What You'll Need
• High-level Workflow
• Detailed Workflow
• Estimated Development Time
• Outcomes After the Integration
• Integration Steps
• Prerequisites: Catalog Integration
• Get Started

Overview

With Fire TV apps, Alexa listens for utterances from users and then converts the utterances into directives. A directive is a set of data and instructions, expressed in JSON, sent from Alexa to your Lambda function in the cloud.

For example, when a user says "Play Bosch," Alexa converts this into a search directive that has a specific JSON structure. Other types of directives include play directives, channel change directives, and more. Alexa does the work of mapping the user's utterances into the formal directives that can be processed programmatically.

The Video Skill API then delivers these directives to your Lambda code on AWS. Lambda is an AWS service that runs code in the cloud without requiring you to have a server to host the code. Your Lambda code can process the directives from the video skill. Your Lambda then sends actions to your app using Amazon Device Messaging (ADM).

What You'll Need

What you'll need depends on the implementation you choose — whether cloud-side or app-only. You will need the following to integrate the VSK with your Fire TV app:

• AWS Account
• Amazon Developer Account
• Android Studio
• Logo images for your video skill
• Fire TV streaming media app (Android). It's recommended that you initially use the Fire TV sample app to see how the VSK works. Then later, you integrate the VSK into your own app.
• Echo Dot or similar Alexa-enabled device
• Fire TV Stick and a TV, or a Fire TV Edition.
• Node JS and npm (the Node installation includes npm). In this documentation, Node JS is used to convert the Lambda function into a zip package that you upload into AWS Lambda. However, you can use other programming languages (other than Node.js) to write your Lambda code.
• Catalog-integrated media
• Alexa Smartphone App (Android | iOS)

High-level Workflow

At a high-level, to integrate the VSK for your Fire TV app, you first create a video skill in the Alexa Developer Console and then associate it with a Lambda function on AWS. When users interact with your app through voice, Alexa voice services in the cloud convert the user's commands into JSON objects, called directives.

Your video skill sends these directives to your Lambda function. Your Lambda function inspects the request, performs any needed lookups or other processing, and then sends the needed information to your app. The Lambda function uses Amazon Device Messaging (ADM), a push notification service, to communicate with your app. A security profile associated with your app authorizes the communication between your app and ADM.

After receiving the communication from your Lambda function, your app would then show a list of results or initial playback of the requested media as requested.

Detailed Workflow

The previous section explained how the VSK works at a high level. Now let's walk through the video skill workflow with more detail and granularity. The following diagram shows the video skill workflow on Fire TV:

Alexa-enabled device listens for natural language commands

On Fire TV, Alexa listens for natural language commands from users. Supported utterances (phrases that Alexa understands) include search, play, app launch, channel change, and transport control commands. The Alexa-enabled device sends these commands to Alexa in the cloud.

Alexa processes phrases and generates out directives

In the cloud, Alexa processes the user's utterances using automatic speech recognition and converts the speech to text. Alexa also processes the commands with natural language understanding to recognize the intent of the text. (As an app developer, you get all of this language processing and interpretation for free.)

Directives are passed to Lambda through the Video Skill API

The output from Alexa in the cloud, which handles the parsing and interpretation of the user's utterances, is a directive. A directive is a set of data and instructions, expressed as a JSON object. For example, when a user says "Watch Big Buck Bunny," Alexa converts this utterance into a SearchAndPlay directive that has a specific JSON structure, like this:

{
    "directive": {
        "payload": {
            "entities": [
                {
                    "type": "Video",
                    "uri": "entity://provider/program/amzn1.p11cat.merged-video.8a42b984-28c2-5c09-bd24-8d924e004d3f",
                    "value": "Big Buck Bunny",
                    "externalIds": {
                        "hawaii_us": "tt1254207",
                        "ENTITY_ID": "amzn1.p11cat.merged-video.8a42b984-28c2-5c09-bd24-8d924e004d3f",
                        "imdb": "tt1254207",
                        "tms": "MV006850300000"
                    }
                },
                {
                    "type": "Video",
                    "uri": "entity://provider/program/amzn1.p11cat.merged-video.4176eed9-eb18-546a-b934-314f50abe8db",
                    "value": "Big Buck Bunny",
                    "externalIds": {
                        "ENTITY_ID": "amzn1.p11cat.merged-video.4176eed9-eb18-546a-b934-314f50abe8db"
                    }
                },
                {
                    "type": "Video",
                    "uri": "entity://provider/program/amzn1.p11cat.merged-video.1ef9c397-544b-5632-a0d3-9b6439113616",
                    "value": "Big Buck Bunny",
                    "externalIds": {
                        "ENTITY_ID": "amzn1.p11cat.merged-video.1ef9c397-544b-5632-a0d3-9b6439113616",
                        "tms": "12631647"
                    }
                },
                {
                    "type": "Video",
                    "uri": "entity://provider/program/amzn1.p11cat.merged-video.5cdb7c5c-8771-55cd-b552-215e131223f1",
                    "value": "Big Buck Bunny",
                    "externalIds": {
                        "ENTITY_ID": "amzn1.p11cat.merged-video.5cdb7c5c-8771-55cd-b552-215e131223f1"
                    }
                },
                {
                    "type": "Video",
                    "uri": "entity://provider/program/amzn1.p11cat.merged-video.3cabe805-968e-5001-813c-f46b5b1069d7",
                    "value": "Big Buck Bunny",
                    "externalIds": {
                        "ENTITY_ID": "amzn1.p11cat.merged-video.3cabe805-968e-5001-813c-f46b5b1069d7",
                        "tms": "SH023726740000"
                    }
                }
            ]
        },
        "header": {
            "payloadVersion": "3",
            "messageId": "72dfff1c-17df-44c5-acbb-19f491e87609",
            "namespace": "Alexa.RemoteVideoPlayer",
            "name": "SearchAndPlay",
            "correlationToken": "1bb6264b-e248-4087-901e-30c3462082b7"
        },
        "endpoint": {
            "endpointId": "1736bf8bd3091561##amzn1-ask-skill-4c43ae24-ee76-4a78-a189-cc06b64d1be8##development##com-fireappbuilder-android-football-streamz",
            "cookie": {
                "VSKClientVersion": "1.4.5",
                "deviceType": "A2LWARUGJLBYEW",
                "appPackageName": "com.fireappbuilder.android.football.streamz",
                "deviceId": "G070L809716314UB",
                "appName": "Sample Alexa VSK Fire TV App",
                "applicationInstanceId": "amzn1.adm-registration.v3.Y29tLmFtYXpvbi5EZXZpY2VNZXNzYWdpbmcuUmVnaXN0cmF0aW9uSWRFbmNyeXB0aW9uS2V5ITEhYUtudnpaOU1xYlV5aU04NElIdU80a3FYa29lVVFDbE5oa2QzM3FhL3hPUzFMaTNmOXBhTkZPeTVaUmFYK3RaU01Cc2Q4b0U0ZzVkOVdhZDR0TVIyb2UxMitUd3dwL0ZEaGpKMkN3bXJhUnUvNThOa0VCRmg1TzYrVmxGN1ZadWlwWmpZZnhEeU1USW1NY1d2MGZYZVMyVDRjVVZSdGtrMWJoQ1FNWEoyQlpRbVNBUmM1V2R5dG5TWUhJZHNwNHg3TzM0MExwQzh4NlhtZlpJY2lpZS9IZktpM0xDYkNFUHlWUTJYU2ZJdVZXNGk3T0c2T0xpWDVlTkl3YXVVZjAyd3JTWGpHVGJrMHRNYU5DcHQ4NGhBVVphQnBHR3dCclVDWGFjcURrUWhMWnd2WjZtZEhqcjNOSkhHd0RDNWt3UGdDWlVRZXRQUkVNVnNqNldlNWtMZ3A0VDdubUZ6SklSeStiSjkxMFEveVh3PSFINFJjeWd2djlCQ2Q1c3NocEptVU5RPT0"
            },
            "scope": {
                "token": null,
                "type": "BearerToken"
            }
        }
    }
}

In this case, the video skill's catalog is called hawaii_us. The first item in the entities array contains a reference to this media.

The following table lists the kinds of directives you can handle with your Lambda function:

In your Lambda and app, you declare a list of the capabilities you support in your response to the Discover directive. You also list the capabilities in your app when you integrate the Alexa Client Library.

The capabilities you declare determine which directives Alexa will send to your Lambda. For example, if you don't declare ChannelController capabilities, Alexa won't send your Lambda any ChannelController directives.

Lambda processes the directives

Lambda is an AWS service that runs code in the cloud without requiring you to have a server to host the code (serverless computing). Your Lambda function processes these directives delivered by the Video Skill API and then responds with a brief status message.

Your Lambda function can use a variety of programming languages, but the sample Lambda code in this documentation uses Node JS. You are responsible for programming the logic in your Lambda function. In other words, Amazon delivers the directive to your Lambda function, and it is your responsibility to figure out how to incorporate the needed actions in your app.

You have flexibility in the way you process the directives coming to your Lambda. You can process the directives in two main ways:

• You can code your Lambda function to handle the directives entirely within the Lambda function itself. Your Lambda function might need to query backend databases or other services to do lookups to get needed information. After the Lambda performs the needed action or retrieves the right information, Lambda can send the instruction to your app.
• Your Lambda function can pass along the directive directly to your app, and your app can handle the processing of the directive. For example, your app can perform queries or other services to do lookups to get needed information before performing some action, and so on.

Lambda sends instruction to your app through ADM

After your Lambda function processes the incoming directive from Alexa, your Lambda function sends instructions to your Fire TV app through push notifications using Amazon Device Messaging (ADM). Your Fire TV app will incorporate the Alexa Client Library, a Java library that assists with voice-enablement for your app. The library helps Alexa prioritize your app when it is active by providing this context to Alexa. The library also helps with authenticating the app with Alexa for automatic skill enablement.

Alexa will ensure that your ADM Registration ID is included in directives sent to your Lambda function. (If you prefer to use a different push notification architecture and cloud service, you can — see Alternatives to ADM.)

Your app acts on incoming message from ADM

Your app receives the instruction and performs the desired result for the user. For example, the action in your app might be to present a movie title to the customer. Again, your Lambda function can perform the processing of the directive directly, or your app can handle it.

Estimated Development Time

It can take anywhere from several weeks to several months to fully integrate the VSK for your Fire TV app. Assuming that your content is already catalog-integrated, the bulk of the development work for the VSK involves creating logic to handle the incoming directives that your Lambda function receives.

The process for integrating the VSK for your Fire TV app is segmented out into a series of steps. See Integration Steps to begin this process.

You will first complete an integration using a sample app (which already has the VSK integrated) and sample Lambda function. This initial integration will take about half a day (depending on your familiarity with Android, AWS, and the Amazon developer console). This simple integration will allow you to see the directives sent from Alexa to your Lambda function in the cloud. Seeing the directives will give you a better sense of the scope of the implementation.

After you walk through the integration with the sample app, you'll need to perform the same integration steps with your real app.

Outcomes After the Integration

In the first part of the integration process, you'll set up a video skill, sample Fire TV app, and basic Lambda function as a quick way to test out the VSK implementation. This will give you a way to see the video skill in action with Alexa responding to user requests, all interacting with a Fire TV app to find and play video. You'll enable the following interaction on an Alexa-enabled device or app. After starting your app, assuming your video skill is called "Streamz", the following interactions will be supported:

The sample app just has a few video titles in its catalog and doesn't support channel change or more advanced directives. In later steps in the integration process, you will customize your Lambda to handle the directives your app supports.

Integration Steps

The process for integrating the Video Skill API into your Fire TV app is broken into the following steps:

To demonstrate the integration, a sample Fire TV app is included that incorporates the Alexa Client Library, ADM, and has some other logic to parse messages and play matching catalog titles. You aren't required to use this sample app in the implementation steps — you can perform the same steps using your own app. The instructions will present options for both paths (either using the sample app or your real app).

Prerequisites: Catalog Integration

Before you can incorporate the VSK into your app, you must complete catalog integration. See the following to integrate your catalog with Fire TV:

• Catalog ingestion. Catalog ingestion refers to the process of describing your app's media according to Amazon's Catalog Data Format (CDF), which is an XML schema, and regularly uploading your catalog into an S3 bucket. Catalog integration is restricted to apps that have long-form movies or episodic TV shows (such as those listed in IMDb). See Getting Started with Catalog Ingestion for details.
• Universal search and browse integration. Your app must integrate with Universal Search and Browse on Fire TV. This integration (with the Fire TV launcher) allows your catalog file to surface media metadata into Amazon's launcher and global search results — regardless of whether customers have your app installed. When users search for content that your app contains, they'll see it in the search results on Fire TV. Make sure you complete both aspects of catalog integration before continuing on here to implement the VSK.

Get Started

Go to the first step of the integration process by going to Step 1: Create Your Video Skill and Set Up Devices.

Last updated: Mar 03, 2023