RTOpacks Internal Docs¶

Internal technical reference for the RTOpacks platform. Zero trust — Tim and Alex only. Entity: United Central Colleges of Australia Pty Ltd · ABN 59 168 872 535

This site is behind Cloudflare Access. If you're reading this, you're authorised.

How to use this docs system¶

This is a working reference, not an archive.

It exists so the engineer (or AI) about to touch a part of the system can brief themselves before they start. Every long doc here is meant to be read in full when its topic is the topic. Skimming for the right answer in fragmented small docs was the failure mode that kept biting us — it produced wrong-context decisions and silent regressions. The fix is the opposite: fewer documents, each long enough to be self-contained, each named so you know exactly which one to open for which job.

If you're about to do something on this codebase, find the doc that owns that concern and read it first. The "Quick start" table below is the index. If the right doc isn't obvious, check the Time Machine — it's the canonical session opener and tells you where current work and current state live.

The principles behind the structure are at the bottom of this page. They explain why things are organised the way they are, so future maintenance respects the same rules instead of letting the corpus drift back into sprawl.

Quick start — what to read for what task¶

If you're about to…	Read this first
Touch any product surface in the workspace (Studio, People, Record, InstaLearn, Campus, Creator, Marketplace)	The relevant module spec under Workspace Product → Apps, plus Product Overview
Write any SQL against any D1 database	Architecture → Database Reference — covers the SQL conventions, the lex gotcha, the `tga_sync_steps` writer contract, bulk-delete safety
Build or modify a sync worker	Infrastructure → Worker Patterns — chain dispatch, all 8 failure modes, pattern selection guide
Investigate a stuck or failed sync run	Operations → Observatory Guide — step taxonomy, OBS-ALIGN-01 query rules, watchdog behaviour
Add a new database table or run a migration	Architecture → Database Reference + Architecture → Data Architecture (which database holds which table)
Wire up notifications or email pipelines	Infrastructure → Notifications
Touch billing, QuickBooks, or Stripe	Infrastructure → Billing — the canonical billing reference (schema, Stripe, QB, runbooks)
Call an external source-of-truth endpoint (TGA, TEQSA, data.gov.au, VOCSTATS, etc.)	Operations → Source of Truth Connectivity — terse matrix of every endpoint + D1 bulk-write gotcha we've rediscovered; read this BEFORE debugging "the API isn't responding"
Onboard a new external API into the platform	Operations → Standing Rules (EXT-API rule) + create a reference doc under `docs/docs/ops/` before deploy
Re-authenticate the QuickBooks OAuth token	Operations → Standing Rules — full procedure + the rolling-token clarification
Pick up where the last session left off	Time Machine — canonical session opener
Understand a worker's purpose, schedule, or bindings	Workers → Worker Inventory — the canonical inventory, updated whenever a worker changes

The reference triangle (for sync work)¶

If the work touches the sync pipeline — any of the workers in tga-sync, cricos-sync, enrich-sync, teqsa-sync, or tga-ingest — three reference docs hang together as a triangle. Each answers a different question and cross-links to the other two.

                  Architecture → Database Reference
                  "What's the writer contract?
                   What's the lex gotcha?
                   How do I write SQL that doesn't bite me?"
                          /            \
                         /              \
              Infrastructure →            Operations →
              Worker Patterns      ⇄      Observatory Guide
              "How do I chain                "What does the
               a worker?                      Observatory show me?
               What are the                   What does this
               failure modes?                 step row mean?
               Which pattern                  What does the
               do I pick?"                    watchdog do?"

Read all three before adding a new sync worker or touching the chain dispatch logic. Each is ~250–500 lines and self-contained. The whole triangle is ~1,000 lines of dense distilled reference — about 30 minutes of reading. That investment up-front prevents most of the failure modes that have bitten us in the past.

Documentation taxonomy¶

Each top-level section has one job. If you're not sure where something belongs, this is the rule:

Section	Question it answers	Notes
Workspace Product	"What does this product DO? What's the canonical module spec?"	Master product doc + 5 module specs (Studio, People, Record, Radar, InstaLearn). The most-read context for any feature build.
Time Machine	"What happened last session? What's open?"	Session-historical, rewritten at the end of every session. Read first thing when you start.
Architecture	"How is the system designed? What are the conventions?"	Auth, data architecture, database reference. Long-lived design decisions.
Reference	"Standing reference material that doesn't fit elsewhere"	Finance spec, DDRs, policy/legal. Small section by design.
Data Infrastructure / Data	"Where does our data come from? What are the sources?"	Data sources, VOCSTATS, sub-processors.
Workers	"What runs where? What's the inventory?"	Canonical worker + queue + DB + KV inventory with current IDs.
Operations	"How do I operate the system day-to-day?"	Observatory guide, LLM batch processing, standing rules, migration reports, TEQSA API reference.
Standards	"What are the cross-cutting product standards?"	Mode system (LIVE/GUIDED/COMPLIANCE).
Design	"What's the visual design system?"	Foundation document.

Principles¶

These are why the corpus is organised the way it is. They're durable rules — when in doubt, follow them.

1. Distill, don't proliferate¶

Fewer long, identity-clear documents beat a sprawling pile of small ones. Every doc here should be long enough to be self-contained on its topic. If you're tempted to create a new doc for a small concern, instead extend the existing doc that owns that concern. The corpus has ~25 working reference docs; that's roughly the right number. If it grows past 40 without a deliberate restructure, something has drifted.

2. Each doc has one identity¶

You should be able to look at a doc's filename and know exactly what's in it. database.md is the SQL conventions and contracts reference. worker-patterns.md is chain dispatch and failure modes. observatory-guide.md is the operator-facing read on tga_sync_steps. No overlap, no ambiguity, no "where does this go?" guessing.

3. Brief yourself before you touch X¶

Every reference doc here is meant to be read in full before you start work on its topic, not skimmed for an answer mid-task. The cost of ~5 minutes reading a doc is consistently lower than the cost of re-discovering a gotcha that's already documented. Build the habit.

4. Archive shipped briefs by default¶

Implementation briefs are session-scoped task documents. They propose work; they're not the canonical state of the work after it ships. Once a brief's work has shipped, the canonical state lives in the appropriate reference doc (architecture, infrastructure, operations, workers, workspace), not in the brief that proposed the work.

Going forwards, every new brief is filed directly into briefs/archive/ and never added to the mkdocs nav. The brief is a working artefact for the session that produces it; it's preserved as historical context (via git log and the archive folder), not as living reference. Concepts or patterns worth keeping go into the appropriate reference doc as part of the same commit that ships the work.

This rule was added in session 58 after we discovered that the active briefs/ tree had grown to 68 files in the mkdocs nav, most of them shipped historical work that was outranking the actual current state in search results.

5. Stale reference is worse than missing reference¶

If a doc no longer reflects reality, it's actively harmful — it tells the reader the wrong thing with the same confidence as a correct doc. When you change behaviour, update or delete the doc in the same commit. Never let a reference doc fall behind the code it references. If you're not sure whether a doc is still accurate, either bring it up to date or delete it.

6. The Time Machine is for "what happened", reference docs are for "how it works"¶

These two purposes don't mix. The Time Machine grows session by session and gets overwritten as new sessions happen. Reference docs (architecture, infrastructure, operations) describe how the system currently works and stay stable across sessions. A change in behaviour goes into the relevant reference doc; a change in session continuity goes into the Time Machine. If a piece of information lives only in the Time Machine and not in a reference doc, it's about to age out — promote it.

7. Cross-link aggressively¶

Every reference doc should point at its companions. The reference triangle (database / worker-patterns / observatory-guide) is the example: each of the three explicitly references the other two so a reader landing in any one of them gets pointers to the related concerns. Cross-links cost nothing to write and prevent the "I didn't know there was already a doc on that" problem.

8. Briefs should not be the source of truth for anything¶

If you find yourself referencing a brief from a reference doc as if the brief were the canonical source — stop. Promote whatever the brief contains into the reference doc itself (or into a new section), then archive the brief. Briefs in the archive are read only for historical context, never as live reference.

How to maintain this system¶

If you're touching a doc:

Read the existing doc first before you decide whether to extend it or create a new one. Extension is almost always the right answer.
If you're creating a new doc, justify why an existing doc can't absorb the content. The default answer should be "no new doc."
Update cross-references when you move or rename anything. mkdocs build --strict will catch broken links during the build.
Run mkdocs build from docs/ after any change so the rendered docs/site/ mirror is regenerated.
Commit references inline with code changes. A behaviour change without a corresponding doc update is incomplete work.
Delete stale content explicitly. Don't leave superseded sections in place "just in case." If you're not sure whether something is still accurate, that's a sign it should be reviewed and either updated or removed — not preserved out of caution.

Quick links¶

Section	What's here
Workspace Product	Master product doc + 5 module specs
Time Machine	Canonical session opener, current state, open hazards
Architecture	Data + auth + database reference
Infrastructure	Worker patterns, ingest, billing, notifications
Operations	Observatory guide, LLM batch, standing rules, runbooks
Workers	Canonical worker inventory
Reference	Standing references (finance spec, DDRs, policy)
Design	Foundation document
Standards	Mode system

Maintained by Tim and Alex. If you're either of them and something on this page feels wrong, fix it in the same commit as whatever you're working on. The whole point of this system is to stay in sync with reality.