Worker Inventory¶
Automated Data Pipeline Workers¶
| Worker | Database | Cron | Dataflows / Purpose |
|---|---|---|---|
| tga-sync | rtopacks-db | 0 16 * * SAT — Sun 2am AEST |
TGA weekly delta sync |
| cricos-sync | rtopacks-db | 0 18 1 * * — 1st/mo 4am AEST |
CRICOS monthly refresh |
| ops-tender-sync | ops-db | 0 8 * * * + 0 10 * * SAT |
AusTender ATM daily + OCDS weekly |
| stats-cache | KV + rtopacks-db | 0 */6 * * * — every 6hr |
ABS + TGA homepage stats |
| qual-enrichment | rtopacks-db | 0 17 * * * — daily 3am AEST |
Qual pre-aggregation |
| abs-monthly-sync | abs-db | 0 20 * * TUE — Tue 6am AEST |
LF, LF_AGES, LF_HOURS, LF_UNDER, LF_EDU |
| abs-quarterly-sync | abs-db | 0 21 1 2,5,8,11 * |
JV, WPI, LABOUR_ACCT_Q |
| abs-annual-sync | abs-db | 0 22 15 12 * — 15 Dec 8am AEST |
AWE, ABS_LABOUR_ACCT |
Infrastructure Workers¶
| Worker | Database | Cron | Purpose |
|---|---|---|---|
| traffic-snapshot | ops-db | 0 2 * * * — daily 12pm AEST |
Zone traffic stats to D1 |
| reg-intel | rtopacks-db | 0 3 * * * — daily 1pm AEST |
Revision alerts (Phase 1 seed) |
Databases¶
| Database | ID | Region | Purpose |
|---|---|---|---|
| rtopacks-db | 334ac8fb-9850-48c0-9da0-b56c55640e98 |
OC (Sydney) | RTOpacks world data |
| ops-db | 00daba3d-2d65-4ae2-b85a-e56d25ec2b02 |
OC (Sydney) | Ops telemetry + tenders |
| abs-db | 66389f3f-f191-467c-be70-f16d5d55fb22 |
OC (Brisbane) | ABS statistical data |
| licensing-db | 6c2abf4d-b5b9-4f51-968b-9e458554fadb |
OC | NSW occupational licensing |
| engine-db | fb6ddc43-e0b9-4ca8-b1c7-b1d36aa573df |
OC | Engine state |
| microcredentials-db | 3924412d-62ad-484b-8366-387056646020 |
OC (Sydney) | Micro-credentials |