Tool Usage Metrics

The tool_usage_metrics mart presents metrics regarding the usage of various LTI tools.

• BQ Prod Dataset Locations

• Interactive Mart Dependency Diagram

• Schema

• Total Events

• Earliest/Latest Event Time

BQ Prod Dataset Locations

• mart_general

Interactive Mart Dependency Diagram

The following visualization shows the construction of this data mart defined in the UDP marts repository. More information on the repository and diagram can be found on this marts page.

Click here to open the interactive chart.

mart_general.tool_usage_metrics

Schema

mart/general/tool_usage_metrics

Field name	Type	Description
ed_app_id	STRING	The EdApp ID of the tool.
total_events	INTEGER	The total number of events associated with the tool.
total_events_1hour	INTEGER	The total number of events for the tool in the most recent 1 hour.
total_events_6hour	INTEGER	The total number of events for the tool in the most recent 6 hours.
total_events_12hour	INTEGER	The total number of events for the tool in the most recent 12 hours.
total_events_day	INTEGER	The total number of events for the tool over the past day.
total_events_week	INTEGER	The total number of events for the tool over the past week.
total_events_month	INTEGER	The total number of events for the tool over the past month.
total_events_year	INTEGER	The total number of events for the tool over the past year.
earliest_event_time	DATETIME	The overall earliest event time associated with the tool.
latest_event_time	DATETIME	The overall latest event time associated with the tool.
earliest_event_time_1hour	DATETIME	The earliest event time for the tool over the most recent 1 hour.
latest_event_time_1hour	DATETIME	The latest event time for the tool over the most recent 1 hour.
earliest_event_time_6hour	DATETIME	The earliest event time for the tool over the most recent 6 hours.
latest_event_time_6hour	DATETIME	The latest event time for the tool over the most recent 6 hours.
earliest_event_time_12hour	DATETIME	The earliest event time for the tool over the most recent 12 hours.
latest_event_time_12_hour	DATETIME	The latest event time for the tool over the most recent 6 hours.
earliest_event_time_day	DATETIME	The earliest event time for the tool over the past day.
latest_event_time_day	DATETIME	The latest event time for the tool over the past day.
earliest_event_time_week	DATETIME	The earliest event time for the tool over the past week.
latest_event_time_week	DATETIME	The latest event time for the tool over the past week.
earliest_event_time_month	DATETIME	The earliest event time for the tool over the past month.
latest_event_time_month	DATETIME	The latest event time for the tool over the past month.
earliest_event_time_year	DATETIME	The earliest event time for the tool over the past year.
latest_event_time_year	DATETIME	The latest event time for the tool over the past year.
low_hourly_events_flag	BOOLEAN	Flag indicating low event volume.

Total Events

The total_events_x fields present the total number of events associated with a given tool for a certain time frame. The time frames included in this mart are all time (2021 to the current date), the past year, the past month, and the past day. Each metric is defined by counting the number of events for the tool in the given time frame. The timeframe is defined using the DATETIME_SUB function in BigQuery.

Earliest/Latest Event Time

The earliest_event_time_x and latest_event_time_x fields respectively present the earliest and latest event times associated with a given tool for a certain time frame. The time frames included are the same as for the total_events_x fields, and defined using the same BigQuery function. The earliest_event_time_x field is defined as the minimum event time for the tool over the given time frame, and the latest_event_time_x field is defined as the maximum event time for the tool over the given time frame.

Low Hour Events Flag

This field helps indicate if a tool outage may be occurring. The most recent hourly volume of events per tool is compared against typical hourly usage for that tool. There are two tiers of comparison:

1. For tools with Caliper events going back at least 1 year, the comparison is against the hourly event volume distribution for the same week number in previous years. For example, let's say we are currently in week 40 of the year, and the most recent hour of events is from 11am - 12pm. We will compare against the 11am - 12pm hourly event distributions in previous years at week 40 for the tool.

2. For tools with Caliper events for less than 1 year, the comparison is against the hourly event volume distribution for the past 6 weeks. For the same 11am - 12pm example, if there is not at least a year of Caliper events to compare against, the comparison distribution is the last 6 weeks of 11am - 12pm event volumes for that tool.

There are three values the flag can take:

• 1 - the flag has been set, indicating low event volume

• 0 - the flag has not been set, indicating typical, expected event volume.

• Null - the flag can't be computed. For tools with less than 6 weeks of data, the flag will have a null value. Tools must also have an average hourly volume of at least 1000 events for the flag to be non-null.

Logic for setting the flag value is as follows:

1. If the tool has less than 6 weeks of Caliper events, the flag will be set to Null

2. If the tool has less than 1000 hourly Caliper events on average in the event window (either past year(s) at the current week number or the most recent 6 weeks), the flag will be set to Null.

3. If the window (either past year(s) at the current week number or the most recent 6 weeks) has an average of at least 1000 events, but the most recent hour has 0 events, the flag will be set to 1.

4. If the most recent hour has more than zero events, but is more than 1.5 standard deviations lower than the window's distribution, the flag will be set to 1.

5. Otherwise, the flag is set to 0.