Creating a New Data Stream

These instructions, and a guide to the rest of Live Events/Canvas Data Services, are hosted in the Canvas Community and are found here.

1. Click on +ADD button to launch a new subscription form
2. Add Subscription Name - use a distinct name to identify your subscription purpose or type e.g : Blackboard Ally Integration
3. Choose Delivery Method
   • SQS - AWS Simple Queue Services
     • URL - AWS SQS URL
     • Authentication via an IAM User Key and Secret is supported but optional. When using a Key and Secret for your SQS queue, please provide the region.
   • HTTPS - Webhook with JWT signing
     • URL - web service endpoint. Each live event will trigger a POST request to this endpoint.
     • If the "Sign Payload" option is not selected, the POST body will be the live event JSON.
     • If "Sign Payload" is selected, the event body will be a signed JWT with the live event data in the claims. Beta and Production JWKs can be found here. These are rotated monthly; that endpoint returns the previous, current, and next (future) JWK used. Most libraries should be able to match the kid in the JWT header to the relevant JWK to validate the signature.
     • If a customer's HTTPS service experiences an outage, failed events are retried on a fixed schedule and then recorded as failed. They are not retried again automatically. See HTTPS delivery failures for more info.
     • More info is found in this Canvas Community article
4. Select the format of the events:
   • Canvas: A simple JSON payload of the events. See the docs for examples
   • Caliper IMS: A standardized JSON object for representing LMS events. See the docs for examples.
5. Find and select a single or multiple events
6. Save your new data stream

Your new subscription will be listed on the Settings page. You will be able to edit, duplicate or deactivate your new subscription record by using right side kebab menu.

SQS configuration

More info is found in this Canvas Community article

1. In the Amazon Web Services console, open the Simple Queue Service (SQS) console by typing the name in the Services field. When Simple Queue Service displays in the list, click the name.
2. In the Amazon SQS console, click the Create New Queue button
3. Enter a name for the queue. The name of the queue must begin with canvas-live-events.
4. By default, Standard Queue will be selected
   • To create a queue with the default settings, click the Quick-Create Queue button. To configure additional queue parameters, click the Configure Queue button. Note: FIFO Queues are not currently supported.
5. Open Queue Permissions
6. Select the checkbox next to the name of your queue. In the queue details area, click the Permissions tab
7. In the permission details window, select the Allow radio button
8. In the Principal field, enter the account number 636161780776. This account number is required for the queue to receive Live Events data
9. Select the All SQS Actions checkbox
10. Click the Add Permission button

HTTPS delivery failures

When setting up a subscription with an HTTPS webhook, the endpoint is expected to return a 2xx HTTP response code within 15 seconds. Anything else counts as a failed delivery: a non-2xx status, a timeout, or a connection error. Redirects are not followed as successes.

Retry schedule

Failed deliveries are retried up to 6 times, for 7 attempts in total. Each wait grows exponentially and has random jitter, so retries don't arrive in bursts.

Retries take about 32 minutes at the earliest and about 96 minutes at the latest.

If the 7th attempt fails, we stop and record the delivery as failed. We don't retry it again automatically.

Building a reliable endpoint

• Return a 2xx quickly and process the event after responding.
• Make your handler idempotent. A retry can arrive after a slow response that we counted as a timeout, so the same event may be delivered more than once.
• Return a 2xx even for events you ignore. Any other status triggers retries.
• Delivery is not ordered. A retried event can arrive after a newer one.

Subscription deactivation

If an excessive number of events fail after these retries, the subscription will be deactivated, and the creator of the subscription will be notified via email. In other words, if you are notified of a failure, it means we have consistently received invalid/error responses (or no response at all) from your webhook's server for a large number of events. The error threshold is approximately 9,500 timeout errors or 28,000 other errors in a 24 hour period. Potential failures on our side do not trigger disabling of the subscription, nor do intermittent failures which succeed upon retry.

It is the end user’s responsibility to implement logging on their endpoints to alert them to any issues and enable troubleshooting should an issue arise. We are unable to provide detailed information the failures, beyond the type of failures and/or HTTP status code, estimated number of events that failed after exhausting retries, and date of occurrence. Common failures include:

• HTTP 4xx or 5xx errors, suggesting some part of your endpoint's infrastructure has rejected requests due to authorization or load issues, or outages
• Timeout errors, suggesting your endpoint's HTTP server may be down or overloaded, or there is a network misconfiguration. Timeouts occur when failing to receive a response after 15 seconds.
• SSL/certificate errors

Since we only consider events that failed after exhausting retries, often errors can be reproduced by simply curling your endpoint.