Unizin Product Documentation
ProductsSupport and TrainingPolicies
  • Unizin Product Documentation
  • Products
    • Content
      • Unizin Engage
        • eReader User Guide
          • Notes, Highlights, and Citations
          • Appearance Settings
          • Download for Offline
          • eReader Layout
          • Keyboard Shortcuts
          • Navigating Your eBook
          • Print
          • Text to Speech
          • Copy and Paste
          • Creating Flashcards
          • Collaboration and Note Sharing
          • Pearson Titles
        • Institution Support
          • Disabled Student Services / Alt-Format
            • Best Practices for Republishing Course Content
            • Disabled Student Services
            • Requesting eTextbook Files for Accessibility Purposes
            • WCAG 2.0 AA evaluation for Engage
            • WCAG 2.0 AA evaluation for EPUB for Engage
          • Institution's Support Responsibilities
        • Caliper 1.1 sensor
        • Release Notes
          • 2.28.22
          • 2019-09-17
          • 2019-05-29
          • 2.26.8
          • 2.26.0
          • 2.25.0
          • 2.22.0
          • 2.21.6
          • 2.21.5
          • 2.20.8
          • 2.20.5
          • 2.20.3
          • 2.19.1
          • 2.18.0
          • 2.17.0
          • 2.14.0
          • 2.12.0
          • 2.11.0
          • 2.9.0
          • 2.8.3
          • 2016-03-17
          • 2016-02-11
          • 2016-01-28
        • Using Analytics (New)
      • Unizin Order Tool
        • Overview of the User Interface
        • Key Concepts
          • Profiles
          • Ordering periods
          • Coordinator permissions
          • Program administrator permissions
        • Courses & Ordering
          • Course filtering
          • Place an order
          • Add sections to a placed order
          • Edit a placed order
          • Cancel an order
          • Reordering
        • Order History
          • Instructor Order History
          • Coordinator and Program Administrator Order History
          • Order Activity
        • Student Choice
          • Student Choice (Program Administrators)
          • Student Choice (Students)
        • Entitlements
          • Entitlements (Program Administrators)
          • Entitlements (Students)
        • Catalog Tool
        • Schedule of Classes
        • Content Request Tool
        • Order Tool Dashboard
        • Vendor Sandbox Tenant
        • Institution Support
        • Implementation
          • SIS Data Integration
            • 1.0 - SIS Integration
            • 2.0 - SIS Integration
          • SSO integration
          • UI customizations
          • Order Feed
            • 1.0 - Order Feed
            • 2.0 - Order Feed
            • 3.0 - Order Feed
          • Publisher report
          • Final declined offers feed
          • Institutional (SIS) Catalog Import
          • Student Price
          • Historical Entitlements Import
        • Release Notes
          • Order Tool Bug Fixes and Enhancements
          • Order Tool Bug Fixes
          • Order Tool Accessibility Improvements
          • Order Feed Improvements
          • Content Request Form Update and Minor Bug Fix
          • Flat Markup Fee Update
          • Ordering Email Receipt Update & Minor Bug Fix
          • Bug Fix for Public Catalog Feature
          • Catalog Search Enhancements
          • Reordering Reminder Email Notifications
          • UX Improvements & Minor Bug Fixes
          • Historical Entitlements Import
          • Student Prices
          • Reordering Feature
          • Email Enhancements
          • Ordering Enhancements
          • Bug Fix for the Institutional Catalog Import
          • Bug Fix for the Final Declined Offers Feed (FDOF)
          • Order Activity Feature and Other Enhancements
          • Bug Fixes for Order History and Report an Issue Features
          • Public catalog feature
          • Minor Bug Fixes for Ordering and Student Choice
          • Entitlements Production Release, Bug fixes, and Minor updates
          • Minor Updates and Bug Fixes for Ordering Workflows
          • Catalog Search Optimization
          • Student Choice
          • Archive Terms Feature and Integration Improvements
          • Introduces the Program Administrator role, Catalog Tool, and Schedule
          • User interface updates and improvements
          • Order feed improvements
          • Order history, UI enhancements
          • Email notification upgrades, UI improvements
          • Order feed changes
          • New features for Course coordinators and upgrades to the UI
          • Changes to the Term, Course, and Section models; introduces a Session
          • Bug fixes, import improvements, and validation improvements
          • Tracking Order History
          • Publisher Reporting
          • Fixes the order feed, automates SIS data importing, and automates the generation of order feed repor
    • Data & Analytics
      • Unizin Data Platform
        • Key concepts
          • Platform overview
          • Data categories
          • Data models
          • Loading schemas
          • Keymap
        • Unizin Common Data Model
          • Academic structures (ERD)
          • Learners (ERD)
          • Course structures (ERD)
          • Course resources (ERD)
          • Learner activities (ERD)
          • Quizzes (ERD)
          • Social (ERD)
          • Course outcomes (ERD)
        • System overview
          • Context data pipeline
            • Context data ingress
            • Batch-ingest application
            • Batch-ingest db server
            • Context store
          • Event data pipeline
            • UDP Caliper endpoint
            • Approval process for implementing Caliper compliant tools
            • UDP Event enricher
            • Event store
        • Data stores
          • Data lake
            • UDP Context store
            • UDP Event store
              • Accessing the Event store
              • Expanded table
                • Expanded table: Canvas edApp mapping
            • Synthetic Data [beta]
              • Viewing Synthetic Data datasets within the BigQuery UI
              • Query Synthetic Data via client libraries
          • Data marts
            • UDP Distributions
            • Interaction sessions
            • Learning Environment Organization
            • File Interaction
            • Last Activity
            • Long Inactivity
            • Course Status
            • Daily Course Grade Record
            • LTI Tool Use
            • LMS Tool Use
            • Tool Usage Metrics
            • Links
            • Taskforce
              • Level 1 Aggregated
              • Level 2 Aggregated
              • Level 2 Course Weekly Distribution Summary
              • Student Term Profile
              • Course Profile
            • Student Activity Score
              • Student Course Metrics
              • Student Course Section Metrics
              • Final
              • Course Final
              • Course Section Final
        • Data integrations
          • Context data integration
            • Loading schema
            • Keymap support
            • Manifest file
            • File requirements
            • Integration mechanics
          • Event data integration
          • SIS data integration
          • LMS data integration
            • Instructure Canvas
        • Release Notes
          • UDP Marts Release Notes
            • 1.0.83
            • 1.0.80
            • 1.0.79
            • 1.0.78
            • 1.0.77
            • 1.0.72
            • 1.0.67
            • 1.0.58
            • 1.0.51
            • 1.0.44
            • 1.0.42
            • 1.0.32
            • 1.0.31
            • 1.0.0
            • Level 2 Taskforce data marts now available
          • 2.0.167
          • 2.0.152
          • 2.0.138
          • 2.0.137
          • 2.0.113
          • 2.0.112
          • 2.0.111
          • 2.0.110
          • 2.0.99
          • 2.0.98
          • 2.0.83
          • 2.0.80
          • 2.0.71
          • 2.0.66
          • 2.0.59
          • 2.0.58
          • 2.0.53
          • 2.0.47
          • 2.0.25
        • Miscellaneous
          • Canvas Data additions, ~Fall 2021
          • Canvas Live Events: from SQS to HTTPS
          • Canvas New Analytics vs. UDP
          • Course Section Enrollment Role Status Mappings
          • Migrating from UDW to UDP
      • Unizin Data Warehouse
        • Implementation Guide
        • Scope of Services
        • Access Provisioning
        • Access Revocation
        • Connecting to the UDW
      • Raw Canvas Data 2
        • Flat Files
        • BigQuery Datasets
    • Hosted Services
      • My Learning Analytics
        • Install MyLA via LTI 1.3
        • Custom configure MyLA
  • Support and Training
    • Professional Development
      • Stepping Stones: A Faculty Development Curriculum for Learning Analytics Use
      • Structured Conversations initiative
    • UDP Self-paced Training
    • Resources Site Broken Links
    • Status Pages
  • Policies
    • General policies
      • Sponsor Teams
      • Browser Support Policy
      • Opt-Out & Invoicing Policy (Order Tool)
    • Support Policy
      • Unizin Engage - SP
      • Unizin Order Tool - SP
      • Unizin Data Platform - SP
      • Unizin Data Warehouse - SP
      • Unizin Data Analysis - SP
      • Pressbooks Hosting - SP
    • Privacy Policy
      • Unizin Engage - PP
      • Unizin Order Tool - PP
      • Unizin Data Platform - PP
      • RStudio service - PP
    • End User License Agreements
      • Unizin Engage - EULA
      • Unizin Order Tool - EULA
    • Terms of Use
      • Unizin Data Platform - ToU
    • Incident Reports
Powered by GitBook
LogoLogo

Unizin Homepage

  • unizin.org

Data & Analytics

  • Unizin Data Platform
  • Unizin Data Warehouse

Content

  • Unizin Engage
  • Unizin Order Tool

Hosted Services

  • My Learning Analytics

Copyright © 2023, Unizin, Ltd.

On this page
  • Internal identifiers
  • External identifiers
  • Primary and composite entities
  • Example scenarios
  • SIS context data integration
  • Learning tool context data integration
  1. Products
  2. Data & Analytics
  3. Unizin Data Platform
  4. Data integrations
  5. Context data integration

Keymap support

PreviousLoading schemaNextManifest file

Last updated 1 year ago

Every UDP Loading schema will specify the data identifiers (or "keys") that a Learning tool, SIS, or LMS must include in its context dataset.

It is not uncommon for data platforms to integrate datasets that include primary and foreign keys in the dataset contents. Indeed, such identifiers are valuable to understand relationships between records in the dataset.

The UDP is unique, however, insofar as it requires that two classes of identifiers be provided for any given record: an internal identifier and an external identifier. In almost every case, internal identifiers are the primary and foreign keys typical of a large dataset generated for batch processing. By contrast, external identifiers are a concept unique to the UDP.

The requirement to provide both classes stems from a central feature of the UDP: the The purpose of the UDP keymap is to associate records from different systems that are about the same thing together.

It is expected that any context dataset provides both internal and external identifiers, as indicated by the loading schema.

Internal identifiers

Internal identifiers are the keys that are native to a particular system. These identifiers are primary keys and foreign keys. The UDP calls them "internal identifiers" because their validity and utility are only internal to the system itself. Internal identifiers are used to define the uniqueness of a record within the context of that system and, when used as foreign keys, to express the relationships between entities in a dataset.

Internal identifiers are required by the UDP's context data import process to ensure the referential integrity of a dataset. Records whose foreign keys fail to refer to an existing primary key in another entity will be dropped.

External identifiers

External identifiers are keys that one system provides (externalizes) to another system and that uniquely identify a record from the first system. It is on the basis of external identifiers that the UDP uses external identifiers to join data from different systems.

External identifiers are usually shared between systems through a data provisioning or launch process. For example, an institution will include SIS identifiers (i.e., external identifiers) in the dataset used to provision the LMS on a nightly basis. As another example, consider that the LMS will pass its own identifiers (i.e., external identifiers) during an LTI launch process to a Learning tool. In both examples, one system is externalizing identifiers from its own system for another system.

It is common for the internal and external identifiers to be distinct. There are many reasons why a system that externalizes its identifiers to another for any given record may not use its actual internal identifiers (primary keys). These may include data modeling constraints (such as a requirement to generate a synthetic identifier), and/or security concerns.

Tools that receive external identifiers from another system may hold on to that system’s external identifiers in a variety of ways. Typically, however, the tool will include an external identifier received from another system as an attribute or property of their own records. For example, an LMS may model its "user" concept to include an "sis_id" attribute.

LMS and Learning tools must provide external identifiers received from other systems in their context datasets.

Primary and composite entities

Most UCDM and Loading schema entities are "primary entities" that represent a fundamental concept or idea. These include entities such as Person, Course section, Academic degree, etc. There also exist "composite entities" that are, as their name suggests, composed of two or more entities. These include entities such as Person-Academic degree or Person-Academic term.

In loading schemas, primary entities will always require both “internal” and “external” identifiers in a context dataset. Composite entities will never require external identifiers. Instead, they will contain foreign keys (which are internal identifiers) to other entities in the context dataset.

Example scenarios

The following two examples illustrate the utility of providing both internal and external identifiers in support of the UDP keymap.

SIS context data integration

It is typical for the Student Information System (SIS) to provision the Learning Management System (LMS) with data for which the SIS is the system of record (e.g., users, academic terms, etc). Almost always, the data provisioning requirements include that the SIS must provide its own identifiers in the dataset used to provision LMS data.

During the provisioning process, the LMS will capture the SIS’s identifiers for a record even as it creates its own new primary keys for the record in its system. Consequently, both systems share a common key (the key included in the SIS dataset) even as they maintain their own primary keys for the same record.

It becomes subsequently possible to join SIS and LMS data based on the LMS’s capture of the identifiers that were provided to it by the SIS.

Learning tool context data integration

Learning tools are integrated into, and launched from, Learning Management Systems. When a Learning tool launch occurs in the LMS, it is almost always by using the IMS Global LTI standard. The LTI standard requires that the LMS pass certain data fields to a Learning tool as part of an LTI launch. Included in these required fields are identifiers for certain records –such as the user and course IDs– that the Learning tool can store for its own use. Such identifiers are effectively external LMS identifiers.

When the UDP consumes data from Learning tools, it will require that the Learning tools UDP Loading schema include the LMS identifiers given to it by the LMS in the LTI launch. Those LMS identifiers are then matched to the LMS's external identifiers, enabling the LMS and Learning tool data to be joined.

For example, the Instructure Canvas LMS can to provision its own internal data for users, academic terms, courses, course sections, and enrollments (among others). The Canvas SIS import process will store the SIS keys (i.e., external identifiers) it receives from the SIS as “SIS IDs.” In its Canvas Data product, for example, the SIS ID that The UDP uses Instructure’s Canvas Data product as its source of LMS context data.

UDP keymap.
import batch data from a Student Information System (SIS)
Canvas receives for a course is called the “sis_source_id.”