> For the complete documentation index, see [llms.txt](https://resources.unizin.org/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://resources.unizin.org/products/data-and-analytics/unizin-data-platform/system-overview/event-data-pipeline.md).

# Event data pipeline

<figure><img src="/files/oyn0Rsnx2MPxiZ3dlIHI" alt=""><figcaption></figcaption></figure>

The UDP's event data pipeline accepts and transforms event data in the 1EdTech Caliper format. The event data pipeline aggregates, computes, and stores event-data and metrics in the UDP's Event store and data marts through a set of loosely-coupled application and infrastructure components.

The ingress point for behavioral data is the UDP Caliper endpoint, a web service that accepts and authenticates Caliper events posted over the web.

Once authenticated and accepted by the UDP Caliper endpoint, events are enriched via the UDP Enricher. The enrichment process incorporates data sourced from the Context store into the event payload. This enables forms of downstream event-data processing that are not otherwise possible or efficient with the data native to an event.

Once enriched, events are routed to various messaging queues (Google PubSub topics) where event-processing services consume events to create and maintain the UDP's Event store and event-based data marts.

The UDP Event store is a date-partitioned Google BigQuery table that serves as the archive for all-time behavioral data. Alongside the UDP Event store are behavioral- and context-derived data marts.

The key components of the Event data pipeline are:

1. [UDP Caliper endpoint](/products/data-and-analytics/unizin-data-platform/system-overview/event-data-pipeline/udp-caliper-endpoint.md), which accepts, authenticates, and routes Caliper events from learning tools
2. [UDP Event enricher](/products/data-and-analytics/unizin-data-platform/system-overview/event-data-pipeline/udp-event-enricher.md), which adds cached context data to streaming behavioral data
3. [UDP Event store](/products/data-and-analytics/unizin-data-platform/system-overview/event-data-pipeline/event-store.md), the data stores that undergird the UDP's data services


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://resources.unizin.org/products/data-and-analytics/unizin-data-platform/system-overview/event-data-pipeline.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
