Unizin Product Documentation
ProductsSupport and TrainingPolicies
  • Unizin Product Documentation
  • Products
    • Content
      • Unizin Engage
        • eReader User Guide
          • Notes, Highlights, and Citations
          • Appearance Settings
          • Download for Offline
          • eReader Layout
          • Keyboard Shortcuts
          • Navigating Your eBook
          • Print
          • Text to Speech
          • Copy and Paste
          • Creating Flashcards
          • Collaboration and Note Sharing
          • Pearson Titles
        • Institution Support
          • Disabled Student Services / Alt-Format
            • Best Practices for Republishing Course Content
            • Disabled Student Services
            • Requesting eTextbook Files for Accessibility Purposes
            • WCAG 2.0 AA evaluation for Engage
            • WCAG 2.0 AA evaluation for EPUB for Engage
          • Institution's Support Responsibilities
        • Caliper 1.1 sensor
        • Release Notes
          • 2.28.22
          • 2019-09-17
          • 2019-05-29
          • 2.26.8
          • 2.26.0
          • 2.25.0
          • 2.22.0
          • 2.21.6
          • 2.21.5
          • 2.20.8
          • 2.20.5
          • 2.20.3
          • 2.19.1
          • 2.18.0
          • 2.17.0
          • 2.14.0
          • 2.12.0
          • 2.11.0
          • 2.9.0
          • 2.8.3
          • 2016-03-17
          • 2016-02-11
          • 2016-01-28
        • Using Analytics (New)
      • Unizin Order Tool
        • Overview of the User Interface
        • Key Concepts
          • Profiles
          • Ordering periods
          • Coordinator permissions
          • Program administrator permissions
        • Courses & Ordering
          • Course filtering
          • Place an order
          • Add sections to a placed order
          • Edit a placed order
          • Cancel an order
          • Reordering
        • Order History
          • Instructor Order History
          • Coordinator and Program Administrator Order History
          • Order Activity
        • Student Choice
          • Student Choice (Program Administrators)
          • Student Choice (Students)
        • Entitlements
          • Entitlements (Program Administrators)
          • Entitlements (Students)
        • Catalog Tool
        • Schedule of Classes
        • Content Request Tool
        • Order Tool Dashboard
        • Vendor Sandbox Tenant
        • Institution Support
        • Implementation
          • SIS Data Integration
            • 1.0 - SIS Integration
            • 2.0 - SIS Integration
          • SSO integration
          • UI customizations
          • Order Feed
            • 1.0 - Order Feed
            • 2.0 - Order Feed
            • 3.0 - Order Feed
          • Publisher report
          • Final declined offers feed
          • Institutional (SIS) Catalog Import
          • Student Price
          • Historical Entitlements Import
        • Release Notes
          • Order Tool Bug Fixes and Enhancements
          • Order Tool Bug Fixes
          • Order Tool Accessibility Improvements
          • Order Feed Improvements
          • Content Request Form Update and Minor Bug Fix
          • Flat Markup Fee Update
          • Ordering Email Receipt Update & Minor Bug Fix
          • Bug Fix for Public Catalog Feature
          • Catalog Search Enhancements
          • Reordering Reminder Email Notifications
          • UX Improvements & Minor Bug Fixes
          • Historical Entitlements Import
          • Student Prices
          • Reordering Feature
          • Email Enhancements
          • Ordering Enhancements
          • Bug Fix for the Institutional Catalog Import
          • Bug Fix for the Final Declined Offers Feed (FDOF)
          • Order Activity Feature and Other Enhancements
          • Bug Fixes for Order History and Report an Issue Features
          • Public catalog feature
          • Minor Bug Fixes for Ordering and Student Choice
          • Entitlements Production Release, Bug fixes, and Minor updates
          • Minor Updates and Bug Fixes for Ordering Workflows
          • Catalog Search Optimization
          • Student Choice
          • Archive Terms Feature and Integration Improvements
          • Introduces the Program Administrator role, Catalog Tool, and Schedule
          • User interface updates and improvements
          • Order feed improvements
          • Order history, UI enhancements
          • Email notification upgrades, UI improvements
          • Order feed changes
          • New features for Course coordinators and upgrades to the UI
          • Changes to the Term, Course, and Section models; introduces a Session
          • Bug fixes, import improvements, and validation improvements
          • Tracking Order History
          • Publisher Reporting
          • Fixes the order feed, automates SIS data importing, and automates the generation of order feed repor
    • Data & Analytics
      • Unizin Data Platform
        • Key concepts
          • Platform overview
          • Data categories
          • Data models
          • Loading schemas
          • Keymap
        • Unizin Common Data Model
          • Academic structures (ERD)
          • Learners (ERD)
          • Course structures (ERD)
          • Course resources (ERD)
          • Learner activities (ERD)
          • Quizzes (ERD)
          • Social (ERD)
          • Course outcomes (ERD)
        • System overview
          • Context data pipeline
            • Context data ingress
            • Batch-ingest application
            • Batch-ingest db server
            • Context store
          • Event data pipeline
            • UDP Caliper endpoint
            • Approval process for implementing Caliper compliant tools
            • UDP Event enricher
            • Event store
        • Data stores
          • Data lake
            • UDP Context store
            • UDP Event store
              • Accessing the Event store
              • Expanded table
                • Expanded table: Canvas edApp mapping
            • Synthetic Data [beta]
              • Viewing Synthetic Data datasets within the BigQuery UI
              • Query Synthetic Data via client libraries
          • Data marts
            • UDP Distributions
            • Interaction sessions
            • Learning Environment Organization
            • File Interaction
            • Last Activity
            • Long Inactivity
            • Course Status
            • Daily Course Grade Record
            • LTI Tool Use
            • LMS Tool Use
            • Tool Usage Metrics
            • Links
            • Taskforce
              • Level 1 Aggregated
              • Level 2 Aggregated
              • Level 2 Course Weekly Distribution Summary
              • Student Term Profile
              • Course Profile
            • Student Activity Score
              • Student Course Metrics
              • Student Course Section Metrics
              • Final
              • Course Final
              • Course Section Final
        • Data integrations
          • Context data integration
            • Loading schema
            • Keymap support
            • Manifest file
            • File requirements
            • Integration mechanics
          • Event data integration
          • SIS data integration
          • LMS data integration
            • Instructure Canvas
        • Release Notes
          • UDP Marts Release Notes
            • 1.0.83
            • 1.0.80
            • 1.0.79
            • 1.0.78
            • 1.0.77
            • 1.0.72
            • 1.0.67
            • 1.0.58
            • 1.0.51
            • 1.0.44
            • 1.0.42
            • 1.0.32
            • 1.0.31
            • 1.0.0
            • Level 2 Taskforce data marts now available
          • 2.0.167
          • 2.0.152
          • 2.0.138
          • 2.0.137
          • 2.0.113
          • 2.0.112
          • 2.0.111
          • 2.0.110
          • 2.0.99
          • 2.0.98
          • 2.0.83
          • 2.0.80
          • 2.0.71
          • 2.0.66
          • 2.0.59
          • 2.0.58
          • 2.0.53
          • 2.0.47
          • 2.0.25
        • Miscellaneous
          • Canvas Data additions, ~Fall 2021
          • Canvas Live Events: from SQS to HTTPS
          • Canvas New Analytics vs. UDP
          • Course Section Enrollment Role Status Mappings
          • Migrating from UDW to UDP
      • Unizin Data Warehouse
        • Implementation Guide
        • Scope of Services
        • Access Provisioning
        • Access Revocation
        • Connecting to the UDW
      • Raw Canvas Data 2
        • Flat Files
        • BigQuery Datasets
    • Hosted Services
      • My Learning Analytics
        • Install MyLA via LTI 1.3
        • Custom configure MyLA
  • Support and Training
    • Professional Development
      • Stepping Stones: A Faculty Development Curriculum for Learning Analytics Use
      • Structured Conversations initiative
    • UDP Self-paced Training
    • Resources Site Broken Links
    • Status Pages
  • Policies
    • General policies
      • Sponsor Teams
      • Browser Support Policy
      • Opt-Out & Invoicing Policy (Order Tool)
    • Support Policy
      • Unizin Engage - SP
      • Unizin Order Tool - SP
      • Unizin Data Platform - SP
      • Unizin Data Warehouse - SP
      • Unizin Data Analysis - SP
      • Pressbooks Hosting - SP
    • Privacy Policy
      • Unizin Engage - PP
      • Unizin Order Tool - PP
      • Unizin Data Platform - PP
      • RStudio service - PP
    • End User License Agreements
      • Unizin Engage - EULA
      • Unizin Order Tool - EULA
    • Terms of Use
      • Unizin Data Platform - ToU
    • Incident Reports
Powered by GitBook
LogoLogo

Unizin Homepage

  • unizin.org

Data & Analytics

  • Unizin Data Platform
  • Unizin Data Warehouse

Content

  • Unizin Engage
  • Unizin Order Tool

Hosted Services

  • My Learning Analytics

Copyright © 2023, Unizin, Ltd.

On this page
  • Authentication
  • Token-based authentication
  • Token Generation
  • Responses
  • HTTP status codes
  • Response body
  • Validation
  • HTTP request errors
  • Caliper envelope errors
  • Caliper event errors
  1. Products
  2. Data & Analytics
  3. Unizin Data Platform
  4. System overview
  5. Event data pipeline

UDP Caliper endpoint

PreviousEvent data pipelineNextApproval process for implementing Caliper compliant tools

Last updated 8 months ago

The UDP Caliper endpoint is a web service that captures behavior data emitted by learning applications. It runs in a Kubernetes cluster and is responsible for accepting, authenticating, and validating behavioral data that conforms with the IMS Global Caliper standard.

The UDP Caliper endpoint supports a standard method of posting behavioral event stream data to the UDP, which is described below. The UDP Caliper endpoint also supports custom event stream integrations with certain Learning Management Systems, such as the Canvas LMS, which are not described below (see our documentation on for more information).

The UDP installation process will automatically configure the web service to be exposed on the public web behind a user-friendly hostname. During the installation process, you will specify a hostname for the service.

Authentication

The UDP Caliper endpoint authenticates the identity of every message posted to its web service.

Token-based authentication

The UDP Caliper endpoint uses token-based authentication, meaning that learning tools posting events to the UDP Caliper endpoint must include a valid token in its HTTP header request. If a Caliper event envelope is posted without a valid authentication token, it will be rejected by the Caliper endpoint.

A token-based authentication method is recommended by the Caliper 1.1 specification. Per the specification, a Caliper sensor (which emits events):

Token-based authentication

SHOULD ... set the Authorization request header field using the "Bearer" authentication scheme described in RFC 6750, Section 2.1. The b64token authorization credential sent by a Sensor MUST be one the Endpoint can validate although the credential MAY be opaque to the emitting Sensor itself.

An example header for a Caliper message sent over HTTPS would therefore look like this:

POST /caliper-endpoint HTTP/1.1
Host: server.example.com
Authorization: Bearer [some_granted_token]

Every time an event is posted to the UDP Caliper endpoint, the "Authorization:" header in the request is inspected and a token is extracted. The token is then compared to a list of configured tokens, called the UDP token map, to determine if the event is valid.

Token Generation

Each token is a unique 64-character hexadecimal string and should be treated as confidential as a password. Unizin will provide tokens securely to the requester.

For reference, an example of a token would look like this: a1b2c3d4e5f67890abcdef1234567890abcdef1234567890abcdef1234567890.

Responses

The UDP Caliper endpoint implements a set of behaviors that enable a learning tool’s Caliper sensor to understand whether its events have been authenticated and accepted by a UDP instance. Furthermore, the UDP Caliper endpoint implements a set of debugging behaviors that can be useful for implementing your Caliper sensor.

In a production environment, we recommend that the Caliper endpoint is run without debugging mode turned on. Keeping the responses terse enables the UDP Caliper endpoint to be more performant at scale. We call these "common" responses (see below).

In a pre-production environment, it is likely useful to run the UDP Caliper endpoint in debug mode (see below). In this mode, the responses are more informative and intended to aid in debugging your Caliper event stream integration.

Finally, whether running in production or debug mode, the UDP Caliper endpoint's response will always, if the events are authenticated, describe how an event was routed for processing.

HTTP status codes

The endpoint may return different status codes depending on the results of authentication and validation.

HTTP request errors:

  • 401 - Unauthorized

  • 415 - Unsupported Media Type. The JSON payload missing or malformed.

Caliper conformance errors:

  • 400 - Non-conformant envelope, or all events non-conformant (distinct from 202 below).

  • 422 - Non-conformant envelope dataVersion. NOTE: A bad dataVersion will always cause a 422 status.

Success:

  • 200 - Success! A valid HTTP request, conformant envelope, all events conformant.

  • 202 - Partial Success. Valid request, conformant envelope, but at least one non-conformant event along with other conformant events.

Failure:

  • Above 500 - the error is unknown and/or the request timed out.

Response body

For any given request, the Caliper endpoint will respond in one of two styles.

Common response

When run in production mode, the Caliper endpoint will always and only return a common response. In debug mode, a common response is given if and only if there is a validation error on the HTTP request itself (e.g., an invalid authentication token is used). In such cases, the Caliper endpoint rejects the request without inspecting the Caliper envelope or its event(s).

{
  "error": "Unauthorized",
  "message": "Bearer token missing or not recognized"
}

Debug response

In debug mode, the Caliper endpoint will always return a verbose response if the HTTP request is valid. To elicit a response from the Caliper endpoint in debug mode, include a request header called X-DEBUG and set its value to TRUE.

Enabling X-DEBUG will cause your events NOT to be accepted. The endpoint will just return a response if the request is valid.

A verbose response is divided into two sections.

The accepted_events attribute contains information about all of the conformant events that were accepted and to which processing they were routed (these are called "topics"). An errors attribute contains information about the Caliper envelope and all of the events that did not conform with Caliper.

 {
  'accepted_events': [
  {
    'id': 'urn:uuid:b46f370c-79e0-433f-9f2b-2d750155e338',
    'topics': { '': '', ... }
  }, …
  ]
  'errors': {
    'envelope': [ ],
    'events': [
      {
        'id': 'urn:uuid:b46f370c-79e0-433f-9f2b-2d750155e337',
        'index': 0,
        'errors': ['Event `actor` attribute must an object or ' 'string.'],
      }, …
    ]
  }
}

Validation

The UDP Caliper endpoint only accepts valid HTTP requests with content-type of JSON, since JSON is the format for Caliper events. The UDP Caliper endpoint will partially validate that the event payload conforms with certain parts of the IMS Global Caliper standard. The UDP Caliper endpoint does not fully validate Caliper conformance.

When a request to the UDP Caliper endpoint, a Caliper event envelope, or a Caliper event is invalid, the UDP will return error messages describing the nature of the failure. Note that these error messages are only returned if the UDP Caliper endpoint is running in debug mode.

HTTP request errors

The following validation requirements exist for the HTTP requests to the UDP Caliper endpoint. If any of them are not met, a corresponding error message is returned.

Requirement
Error message

An authentication token is present and valid.

Bearer token missing or not recognized

The content-type is JSON.

Unsupported Media Type", "message": "application/json payload expected.

Caliper envelope errors

Requirement
Error message

The "sensor" attribute is present.

Event envelope must include a `sensor` attribute.

The "sendTime" attribute is present and in ISO 8601 format.

Event envelope `sendTime` attribute must be an ISO 8601 time string formated like "yyyy-mm-ddThh:mm:ss.SSSZ".

The "dataVersion" attribute is present.

The "dataVersion" attribute is valid.

Event envelope `data` attribute must be an array with at least one element.

Caliper event errors

All Caliper events in a Caliper envelope must conform with the Caliper standard. The Caliper endpoint will reject any events that fail the following requirements. The following validation requirements exist for Caliper events. If any of them are not met, a corresponding error message is returned.

Requirement
Error message

The "id" attribute is present.

Event `id` must be present. Version 4 UUID is recommended.

The "@context" attribute is present and valid.

The "actor" attribute" is present.

Event `actor` attribute must an object or string.

Note: we recommend defining actor as a hash with an id and type attribute. Alternatively, actor may be defined as a string.

The "action" and "type" attributes are present.

"Event `type` and `action` must be present, and `action` must be valid for `type`.

The "eventTime" is present and valid ISO 8601 format.

Event `eventTime` attribute must be an ISO 8601 time string formatted like "yyyy-mm-ddThh:mm:ss.SSSZ".

To request tokens, please email Support at . We highly recommend using tokens exclusively for a single tool integration (such as Kaltura, TopHat, Macmillan iClicker, Canvas Live Events, etc.) and not reusing them across different tools or UDP instances.

All events posted to the Endpoint must be delivered in a Caliper envelope. Caliper envelopes must conform to the. The following validation requirements exist for Caliper envelopes. If any of them are not met, a corresponding error message is returned.

Event envelope `dataVersion` attribute must be a version like "".

Event `@context` must be a version like "".

LMS data integrations to the UDP
support@unizin.org
version 1.1 of the IMS Global Caliper standard
http://purl.imsglobal.org/ctx/caliper/v1p1
http://purl.imsglobal.org/ctx/caliper/v1p1