RTOpacks Radar — Product Specification¶

Document ID: RADAR-SPEC-01
Version: 0.3
Status: Living document — captures deployed system state + product intent
Location: docs/docs/workspace/apps/radar.md
Last updated: 4 July 2026

Changelog¶

Version	Date	Changes
0.1	17 Apr 2026	Initial — full signal taxonomy, data model, crawl pipeline, UI surfaces, product positioning
0.2	17 Apr 2026	Enrichment pipeline, 24 promoted columns, compliance signals, link destination resolution, third-party/infrastructure domain split, provenance layer, dual screenshots, enrichment_data JSON schema
0.3	4 Jul 2026	Product architecture ratified: Radar/Marketer split (outward lens vs mirror-plus-hands), four product layers (Register → Feed → Watchlist → Dossier depth), data canonicity ruling (newest-wins, dates visible), append-only substrate law + revision scrubber, freshness model (crawl-on-onboarding + subscriber watchlist + ambient trickle), commercial philosophy (held-depth markers, one subscription, full depth pre-entitlements), CAPTURE-01 named as the feed's engine, public/private data line, ADR-020 tier nomenclature, substrate state from RADAR-SUBSTRATE-RECON-01

How to Use This Document¶

This spec is the full-picture reference for Radar, also known as Field Observer when surfaced as a standalone product. It documents what the system is, what it observes, how it observes it, and what it does not do. Every build brief that touches Radar references this document as its source of truth.

This document does not constitute a brief. Briefs are scoped, sequenced, and issued separately. The reason for this is the same as People: the full system described here is too large to have been built in one pass, and some parts are still planned. What should never happen is a brief that contradicts this document, or a build that forecloses something described here.

Alex: read this document before touching any Radar code. The briefs tell you what to build. This document tells you why, and what "Radar" actually is.

What Radar Is¶

Radar is RTOpacks' outward-facing intelligence lens. It assembles a structured dossier on every registered training organisation in Australia — their digital presence, their technology stack, their government registration status, and their public reputation — using only publicly observable data.

Radar and Marketer share one substrate — the footprint database — pointed in opposite directions. Radar is the telescope aimed at everyone-but-you: the register, the sector, competitors. Marketer is the same telescope turned on the customer's own RTO, plus the hands to act on what it shows (GBP control, campaigns, remediation — see WS-PRODUCT-01). The own-org mirror ships under the Marketer tile, not Radar (ratified 2026-07-04). This spec governs the outward lens and the shared observation substrate.

It does not access anything private. It does not authenticate to any system. It does not use AI to draw conclusions. It reads what is already published — sitemaps, DNS records, SSL certificates, government registers, social media profiles — and assembles it faster than any human team could, into a format where it can actually be seen and compared.

The product's core claim, stated plainly: Everything you see here was already public. We just connected the dots.

Who uses it¶

RTOpacks operators (T3 operator tier / T4 administrator — ADR-020; L-numbers in earlier versions of this spec are legacy nomenclature) use Radar to understand the RTO landscape — which organisations are active, which are dormant, what technology they run, how mature their digital presence is.
RTO customers (T4/T4A) get the outward product defined in Product Architecture below: register, sector feed, watchlist, dossier depth. The own-RTO mirror — how you appear from the outside, and its "slight vertigo" disarming move — belongs to Marketer (ratified 2026-07-04) and is documented there and in WS-PRODUCT-01.
Regulatory analysts use the signal layers to identify anomalies — RTOs with expired SSL, no web presence, mismatched entity types, or dormant digital footprints despite active registrations.

What it is not¶

Radar is not surveillance. It does not monitor private communications, access authenticated systems, or track individual behaviour. Every signal it captures is available to anyone with a web browser and the patience to look. Radar's value is not in what it sees — it's in the fact that it sees everything, at the same time, and puts it somewhere useful.

Say this, not that:

Say this	Not that
Public data, assembled	Data harvested
What's on the label	What we discovered
We read your sitemap	We analysed your site
The same information anyone could find	Proprietary intelligence
A mechanical process	An algorithm
We connected the dots	We drew conclusions
No AI	Powered by AI

Product Architecture (ratified 2026-07-04)¶

This section captures the product rulings taken 4 July 2026. It governs the customer-facing expression of Radar; the design brief that executes it is RADAR-WORKSPACE-EXPRESSION-01 (chambered). Where anything below conflicts with earlier sections of this spec, this section wins.

Two tiles, one telescope¶

The Intelligence row's four tiles are one idea pointed in four directions: Radar (competitors and the sector), Landscape (the vendor ecosystem), Market Data (the macro economy), Marketer (yourself). Three of them only look. Marketer is the one tile with hands — it observes the customer's own footprint and acts to improve it, so that the next observation measures the improvement. Radar and Marketer run on the same footprint substrate with opposite filters: everyone-but-you versus only-you.

Because they share the telescope, the measurement loop closes for free: Marketer fixes the listing, launches the campaign, tidies the profile — the next own-org crawl certifies the fix. The instrument that flagged the problem is the instrument that proves the remedy.

The four product layers¶

Radar's customer expression is built as four layers, each standing on the one below. This is also the build order — cheapest and most foundational first.

#	Layer	What it is	Commercial position	Engine / dependency
1	Register	Search any RTO, get its page. The enriched, human-readable mirror of the public record — "TGA with enhancements." The navigation substrate everything else hangs off.	Free. All of it is public record; charging for what government publishes free makes us a tollbooth. The register is the front door, not the product.	Org data already synced; footprint DB already keyed by `rto_code`. Nearest to done.
2	Sector feed	"What moved this week": scope changes, registration changes, regulatory decisions, rendered as a live feed. The Sentinel in action, and the shop window — every item is a tap toward depth.	Headlines included with any workspace subscription. The tap-through (timelines, histories, pattern reads) carries a held-depth marker where not entitled.	CAPTURE-01. TGA exposes no change endpoint anywhere in its 88 API operations (TGA-SWAGGER-VS-REPORTS-DELTA-AUDIT-01, finding C5) — snapshot-and-diff is the only source of temporal intelligence. The feed is CAPTURE-01's output rendered.
3	Watchlist	The subscriber nominates the RTOs they care about; those join the fresh-crawl rotation and their movements surface first. How the customer aims the telescope.	Subscription.	Register (to find RTOs) + freshness model (below).
4	Dossier depth	Full intelligence on any RTO: scope history across years, regulatory decision chains, footprint analysis, the duck-and-shark reads. Time and patterns.	Subscription.	The signal layers and enrichment documented in this spec, plus the append-only revision substrate (below).

The dividing rule, stated once: facts free, patterns paid. Current-state public record is free; time, depth, and assembled insight are the intel product.

Marketer as the demo¶

Every customer gets full depth on themselves through Marketer — the whole footprint machine at maximum resolution, pointed at the one RTO they know intimately. Radar's outward view ships light by default; full outward depth is the paid intel subscription.

The conversion mechanism is verification asymmetry: a customer cannot check whether our intelligence on a competitor is accurate — but they can audit it on themselves, warts and all. When Marketer nails their own footprint, the accuracy transfers: "if it knows this much about me, it knows this much about them." Marketer answers the prospective subscriber's only real question — what would I actually get? — by showing them, on the subject where they are the expert.

Within Marketer itself, the later tier line (not drawn yet) runs between the view (the full-depth mirror — the demo, costs one crawl) and the hands (GBP control, campaign creation, remediation — the value-add layer). View included, hands paid, outward depth subscribed: three products, one substrate. Marketer's action layer follows house disposition — the engine drafts, the human signs; every external publish is tapped by the customer's own person. Each external platform integration (Google, Meta, etc.) is an EXT-API doc before deploy.

Commercial philosophy — held-depth markers, no walls¶

Two failure modes to design out: the tollbooth (charging for public facts) and the taxi meter (per-touch micro-charges and upgrade walls).

One subscription, not micro-transactions. A single Radar intel tier opens everything at that tier. No per-dossier or per-report charges.
Held-depth markers, not paywalls. Entitled data renders fully and usefully. Where deeper holdings exist beyond the customer's plan, the UI states it as quiet inventory — "We hold enhanced intelligence on this RTO — scope history to 2011, 47 footprint signals — not included in your current plan" — one factual line, no wall, no modal, no nag. This is Sentinel posture applied to commerce: we tell you the truth about what we know, including the truth that we know more. The marker line is itself intelligence, which is why it converts.

Full depth first — entitlements deferred¶

Pre-revenue ruling: build at full depth, everything exposed. Inclusion/exclusion lines cannot be drawn around a product that doesn't exist, and lines drawn now would be guesses. The platform has billing (mostly) and identity (ADR-020) but no entitlements layer joining them — that gap is filed as ENTITLEMENTS-MODEL-01, TO-BUILD, triggered alongside client onboarding. One discipline applies now: every Radar/Marketer surface renders depth as a parameter, not a hardcode, and the held-depth marker component exists from day one (pre-entitlements, every tap opens). The tier line later is a config change, not a refactor.

Data canonicity ruling¶

Substrate fact (RADAR-SUBSTRATE-RECON-01, July 2026): the corpus is two-generation. Gen-1 — 12,508 dossiers, 446,610 signals, the rich layer the admin UI renders — is frozen at the 2–3 April 2026 bulk pass. Gen-2 — radar_crawl_results, observations/_current, MAGDA — is fresh to July 2026 but fed by a ~20-RTO/month trickle (~52-year full cycle). Enrichment coverage ~36%.

The ruling: neither generation is canon alone. Newest observation wins per signal, and every rendered observation carries its date, visibly. For most RTOs that means April's picture labelled "observed 2 April 2026" — old but honest. Freshly crawled RTOs show new data wherever it overlaps, with Gen-1 detail persisting for whatever the new pass didn't cover. What is never done: rendering frozen data undated so it reads as current. A paid intel product does not survive that dishonesty, and this house doesn't do it anyway.

Append-only substrate law + the revision scrubber¶

From the canonicity ruling forward: crawls never overwrite; they accumulate. Every crawl is a dated revision; a _current pointer designates the newest. Retro-creating history is impossible — Gen-1 is frozen precisely because April was a single shot — so every crawl from here on is a frame in the film.

The UI expression is a timeline scrubber on the dossier (the Google Street View pattern): drag back through revisions and see the RTO as observed then versus now. The house already holds this pattern — see STUDIO-HISTORY-SCRUBBER-01 — so Radar reuses, not invents. Storage is a non-issue at our crawl volumes (whole radar DB ≈ 332 MB; revisions grow with crawl count, which the freshness model keeps small; screenshots live in R2).

Migration implications (TO-BUILD, must precede or accompany the expression work): the current substrate partially violates the law — radar_crawl_results is keyed rto_code PRIMARY KEY (one row, overwritten per crawl) and screenshot/favicon R2 keys are fixed ({rto_code}/homepage.jpg, overwritten). The HTML archive (radar-html/{rto_code}/{timestamp}.html.gz) is already timestamped and is the model. Crawl results move to revision rows keyed (rto_code, crawled_at); screenshots move to timestamped keys. Per MIGRATION-COMPLETION-DISCIPLINE, the migration brief includes retirement of the overwrite paths.

Note the rhyme, because it is the product's physics: the swagger audit concluded snapshot-and-diff is the only path to temporal intelligence on the TGA side; the scrubber is the same conclusion on the crawl side. The whole intelligence layer is a time machine.

Freshness model¶

Freshness is a per-customer property, not a corpus property — the only version affordable at trickle volumes.

Channel	Trigger	Covers
Crawl-on-onboarding	Customer signs up	Their own RTO — the Marketer demo is always fresh
Watchlist rotation	Subscriber nominates competitors	The RTOs a paying customer actually watches — crawl cost scales with revenue, not with the 12,500-dossier corpus
Ambient trickle	Standing ~20/month rotation	The background corpus, slowly
Bulk pass	Manual, on decision	Available as a tool, not a cadence

This model formally dissolves the old bulk-burst rationale and reframes the Radar-CF migration: the build target is the on-demand crawl path (onboarding + watchlist + feed-item refresh), not a migrated burst.

The public/private line¶

A tempting later move is public register pages (12,500 indexed RTO pages is a serious SEO engine). The line, drawn now on purpose: public pages carry TGA-sourced public-record facts only. Crawl-derived footprint enrichment about non-customers lives behind login. Showing a logged-in subscriber assembled public observations is the product; publishing footprint analysis of other people's businesses to the open web is a different ethics and exposure posture, and it is not taken by accident.

Adjacent finding, filed for the signal roadmap¶

TGA's delivery-notification history endpoints (per-RTO, per-unit, who-delivers-what over time) are confirmed unhoovered high-value (swagger audit, C4). Regulatory-sourced footprint data no crawl can produce — a natural future Layer-1-grade signal family for dossier depth, and eventually the spine of any landscape/comparative product. Filed as ingest-scoping input; no brief auto-opens.

The Dossier Model¶

Every RTO has a dossier — a single record that anchors all intelligence collected about that organisation. The dossier is created automatically when any Radar surface first references an RTO. It persists indefinitely.

Dossier fields¶

Field	Type	Description
`id`	TEXT PK	UUID
`rto_code`	TEXT UNIQUE	The RTO's national code from TGA
`last_crawled_at`	TEXT	ISO timestamp of most recent crawl completion
`crawl_status`	TEXT	Lifecycle: `pending` → `queued` → `crawling` → `complete` / `failed`
`notes`	TEXT	Free-text analyst notes (operator-entered)
`created_at`	TEXT	Auto
`updated_at`	TEXT	Auto

The dossier is a shell. Intelligence lives in satellite tables keyed to rto_code, plus the radar_crawl_results table which holds both sitemap crawl data and enrichment data from the Puppeteer pass.

Signal Layers¶

Radar organises intelligence into five layers, ordered by confidence — from government-reported facts at the top, to social signals at the bottom. Each layer is independently populated, independently timestamped, and independently refreshable.

Layer 1 — Government Reality¶

Confidence: Authoritative
Sources: Australian Business Register (ABR), ASIC, ASQA, ACNC
What it tells you: Is this entity real? Is it active? What kind of entity is it?

Category	Signal Key	Description
`abn`	`abn_value`	The ABN itself
`abn`	`abn_status`	Active / Cancelled / other
`abn`	`abn_status_effective`	Date the status took effect
`abn`	`abr_entity_name`	Legal name as registered with ABR
`abn`	`abr_last_updated`	When ABR last modified this record
`abn`	`entity_type_code`	ABR entity type (company, sole trader, trust, etc.)
`abn`	`entity_description`	Human-readable entity description
`abn`	`gst_status`	GST registration status
`abn`	`gst_registered_date`	When GST was registered
`abn`	`asic_number`	ASIC company number (ACN) if applicable
`abn`	`acnc_registered`	Whether registered with ACNC (charities)
`abn`	`charity_type`	ACNC charity subtype
`abn`	`dgr_endorsed`	Deductible Gift Recipient status
`abn`	`address_state`	Registered business state
`abn`	`address_postcode`	Registered business postcode
`classification`	`entity_type`	Derived entity classification (school, TAFE, university, commercial, NFP, etc.)
`classification`	`non_commercial_flag`	Whether this is a non-commercial entity
`classifier`	`entity_type_abn`	Entity type derived from ABN data (highest precedence)
`classifier`	`risk_flag`	Risk indicators (multiple allowed)
`registration`	`edu_au_anomaly`	Anomaly flags on .edu.au domain usage

Entity type classification precedence: ABN entity type > domain pattern > sweep inference > unknown. The entity badge in the UI follows this precedence chain.

Entity type values: school, tafe, university, government, association, sole_trader, trust, nfp, charity, commercial_rto, unknown_edu, unknown.

Layer 2 — Public Presence¶

Confidence: Observed
Sources: DNS, WHOIS, SSL certificates, HTTP responses, Wayback Machine, TLS handshake (enrichment pass)
What it tells you: Does this organisation have a functioning web presence? How established is it? Is their infrastructure secure?

Category	Signal Key	Source	Description
`domain`	`has_edu_domain`	DNS	Whether the RTO uses a .edu.au domain
`domain`	`edu_website_live`	HTTP	Whether the .edu.au site responds
`domain`	`edu_mx_live`	DNS	Whether .edu.au MX records are active
`domain`	`domain_registered`	WHOIS	Domain registration date
`domain`	`domain_expiry`	WHOIS	Domain expiry date
`domain`	`registrar`	WHOIS	Domain registrar
`domain`	`registrant_privacy`	WHOIS	Whether WHOIS privacy is enabled
`domain`	`nameserver_provider`	DNS	NS provider (Cloudflare, AWS Route53, etc.)
`domain`	`namespace_active`	DNS	Whether the domain actively resolves
`domain`	`dnssec`	DNS	DNSSEC enabled
`domain`	`axfr_permitted`	DNS	Zone transfer permitted (security flag)
`email`	`contact_email`	HTML	Publicly listed contact email
`email`	`email_domain`	HTML	Email domain used
`email`	`email_domain_resolves`	DNS	Whether the email domain resolves
`ssl`	`ssl_issuer`	TLS handshake	Certificate issuer (Let's Encrypt, DigiCert, Sectigo, etc.)
`ssl`	`ssl_valid_to`	TLS handshake	Certificate expiry date (ISO)
`ssl`	`ssl_days_remaining`	TLS handshake	Days until cert expiry (negative = expired)
`ssl`	`ssl_cert_error`	TLS handshake	Certificate error if any (expired, name mismatch, etc.)
`ssl`	`hsts`	HTTP header	HTTP Strict Transport Security enabled
`redirect`	`final_url`	HTTP	URL after all redirects (stale domain detection)
`redirect`	`had_https_redirect`	HTTP	Whether initial HTTP URL redirected to HTTPS
`history`	`wayback_first_snapshot`	Wayback API	Earliest Wayback Machine capture
`history`	`wayback_latest_snapshot`	Wayback API	Most recent Wayback Machine capture
`classifier`	`web_age_years`	Derived	Years since first Wayback snapshot
`classifier`	`ssl_health`	Derived	good / adequate / expiring_soon / critical

Enrichment adds: ssl_issuer, ssl_valid_to, ssl_days_remaining, ssl_cert_error, final_url, had_https_redirect are promoted columns on radar_crawl_results, populated by the local Puppeteer enrichment pass from the TLS handshake. Full SSL details (subject, valid_from, protocol, SAN list note) are in the enrichment_data JSON blob under $.ssl.

Layer 3 — Technical Fingerprint¶

Confidence: Observed / Inferred
Sources: HTTP response headers, HTML source analysis, rendered DOM, network request interception, known platform patterns
What it tells you: What technology runs this site? How sophisticated is it? What compliance signals are visible?

Fingerprint signals¶

Category	Signal Key	Source	Description
`fingerprint`	`page_title`	DOM	HTML `<title>` (promoted column)
`fingerprint`	`meta_description`	DOM	Meta description
`fingerprint`	`meta_keywords`	DOM	Meta keywords (legacy SEO)
`fingerprint`	`og_title`	DOM	OpenGraph title
`fingerprint`	`og_site_name`	DOM	OpenGraph site name
`fingerprint`	`og_description`	DOM	OpenGraph description
`fingerprint`	`og_image_url`	DOM	OpenGraph image URL
`fingerprint`	`twitter_card`	DOM	Twitter card type
`fingerprint`	`twitter_site`	DOM	Twitter @handle
`fingerprint`	`generator`	DOM	CMS generator tag (WordPress, Squarespace, etc.) (promoted column)
`fingerprint`	`server_header`	HTTP	HTTP `Server` header
`fingerprint`	`x_powered_by`	HTTP	HTTP `X-Powered-By` header
`fingerprint`	`robots_meta`	DOM	Robots meta directive
`fingerprint`	`copyright_year`	DOM	Copyright year found in footer
`fingerprint`	`mobile_responsive`	DOM	Viewport meta tag exists and is not fixed-width (promoted column)
`fingerprint`	`canonical_url`	DOM	`<link rel="canonical">` URL
`fingerprint`	`viewport_meta`	DOM	Raw viewport meta content string
`fingerprint`	`wp_rest_api_exposed`	HTTP	WordPress REST API publicly accessible

Analytics and pixel detection¶

Detected via dual strategy: HTML source regex AND observed network requests during page load.

Signal Key	Detection	Description
`gtm` + `gtm_id`	HTML regex (`GTM-*`) + network (`googletagmanager.com`)	Google Tag Manager
`ga4` + `ga4_id`	HTML regex (`G-*`) + network (`google-analytics.com`)	Google Analytics 4
`google_ads` + `google_ads_id`	HTML regex (`AW-*`) + network (`googleadservices.com`)	Google Ads conversion
`meta_pixel` + `meta_pixel_id`	HTML regex (`fbq('init'`) + network (`connect.facebook.net`)	Meta/Facebook pixel
`tiktok_pixel` + `tiktok_pixel_id`	HTML regex (`ttq.load`) + network (`analytics.tiktok.com`)	TikTok pixel
`linkedin_insight` + `linkedin_insight_id`	HTML regex (`_linkedin_partner_id`) + network (`snap.licdn.com`)	LinkedIn Insight
`hubspot_tracking`	HTML regex (`hs-script-loader`) + network (`js.hs-scripts.com`)	HubSpot tracking
`hotjar`	HTML regex (`hotjar.com`) + network (`hotjar.com`)	Hotjar
`intercom`	HTML regex (`intercomSettings`) + network (`intercom.io`)	Intercom
`zendesk_chat`	HTML regex (`zE(`) + network (`zendesk.com`)	Zendesk Chat
`tawk_chat`	HTML regex (`tawk.to`) + network (`embed.tawk.to`)	Tawk.to Chat

Compliance signals¶

Mechanical observations from the rendered homepage. No inference, no model calls. All regex-based or link-presence checks.

Signal Key	Type	Source	Description
`rto_code_on_homepage`	BOOLEAN	Text regex	RTO's own national code visible in page text (promoted column)
`cricos_code_on_homepage`	BOOLEAN	Text regex	CRICOS provider code visible (promoted column)
`cricos_code_value`	TEXT	Text regex	The matched CRICOS code (e.g. `00001K`) (promoted column)
`usi_mentioned`	BOOLEAN	Text regex	"USI" or "Unique Student Identifier" in page text
`privacy_policy_link_present`	BOOLEAN	Link check	`<a>` with text containing "privacy" exists (promoted column)
`privacy_policy_url`	TEXT	Link href	URL of the privacy policy link
`complaints_link_present`	BOOLEAN	Link check	`<a>` with text containing "complaint", "grievance", or "appeals" (promoted column)
`complaints_link_url`	TEXT	Link href	URL of the complaints link
`accessibility_statement_present`	BOOLEAN	Link check	`<a>` with text containing "accessibility" or "wcag" (promoted column)
`accessibility_url`	TEXT	Link href	URL of the accessibility statement
`government_program_mentions`	JSON array	Text regex	Detected mentions of: Smart and Skilled, Free TAFE, Fee-Free TAFE, VET Student Loans, Australian Apprenticeships, JobTrainer, User Choice, Construction Blueprint, Skills First, Higher Level Skills
`third_party_enrolment_system`	BOOLEAN	Link resolution	"Enrol" or "Apply" links resolve to known external platforms (promoted column)

Layer 4 — Integration Signals¶

Confidence: Observed / Inferred
Sources: MX records, DNS TXT records, HTML source, subdomain fingerprinting, network request interception, link destination resolution
What it tells you: What third-party services does this organisation depend on?

Category	Signal Key	Description
`mail`	`mail_platform`	Email platform (Google Workspace, Microsoft 365, etc.)
`mail`	`email_domain_mail_platform`	Mail platform on the email domain specifically
`mail`	`spf_present`	SPF record exists
`mail`	`dkim_present`	DKIM record exists
`mail`	`dmarc_present`	DMARC record exists
`mail`	`dmarc_policy`	DMARC policy (none, quarantine, reject)
`lms`	`lms_platform`	LMS platform detected (from signals table)
`lms`	`lms_url`	URL of detected LMS
`lms`	`lms_hosting_same_as_main`	Whether LMS is on same infrastructure
`lms`	`lms_platform_detected`	LMS detected via link resolution (promoted column on crawl_results)
`sms`	`sms_platform_detected`	Student management system detected via link resolution (promoted column)
`integration`	`third_party_service`	Named third-party service detected
`classifier`	`email_maturity`	Derived: mature / moderate / basic / none

Layer 5 — Human Signals¶

Confidence: Inferred (lowest)
Sources: Social media platforms, Seek job listings, Wikipedia
What it tells you: Is this organisation active in the public sphere? Is it hiring?

Category	Signal Key	Description
`social`	`social_facebook`	Facebook URL
`social`	`social_linkedin`	LinkedIn URL
`social`	`social_instagram`	Instagram URL
`social`	`social_twitter`	Twitter/X URL
`social`	`social_youtube`	YouTube URL
`social`	`social_tiktok`	TikTok URL
`social`	`facebook_blocked`	Whether Facebook profile was blocked/private
`social`	`linkedin_blocked`	Whether LinkedIn profile was blocked/private
`seek`	`seek_active_listings`	Number of active Seek job listings
`seek`	`seek_locations`	Locations in Seek listings
`reputation`	`wikipedia_present`	Whether a Wikipedia article exists
`reputation`	`wikipedia_title`	Wikipedia article title
`reputation`	`wikipedia_description`	Wikipedia article summary
`classifier`	`hiring_signal`	Derived: active_hiring / dormant

Confidence Tiers¶

Every signal carries a confidence level. Three tiers:

Tier	Meaning	Example
Authoritative	Government-reported, citable, legally binding	ABN status from ABR
Observed	Directly detected by automated sweep, timestamped, repeatable	SSL certificate issuer from HTTPS handshake
Inferred	Derived from authoritative or observed data — a reasonable conclusion, not a direct observation	Hosting tier derived from IP/ASN lookup

The UI distinguishes these with colour-coded confidence pills. Authoritative signals are presented as facts. Inferred signals are presented as inferences. This distinction is not cosmetic — it determines whether the signal is citable in a regulatory context.

Stack Profile¶

Separate from the signal layers, each RTO has a stack profile — a table of detected technologies organised by surface area. This is the "what are they running?" view.

Surfaces¶

Surface	What it covers
`frontend`	CMS, static site generator, JavaScript framework
`lms`	Learning management system (Canvas, Moodle, Blackboard, etc.)
`email`	Email platform (Google Workspace, Microsoft 365, Zoho, etc.)
`hosting`	Cloud provider, CDN, shared hosting
`cdn`	Content delivery network
`analytics`	Analytics and tracking tools
`payment`	Payment processing (Stripe, PayPal, etc.)
`student_portal`	Student management system / portal
`other`	Anything that doesn't fit above

Each entry records: surface, technology name, description, cost range estimate, vendor URL, confidence level, and observation timestamp.

Crawl Pipeline¶

Data flow¶

rto-nrt-db (read) → radar-crawl worker → radar-db (write) + R2 (screenshots)

The crawl pipeline is a Cloudflare Worker (radar-crawl) that processes RTOs in batches. For each RTO:

Resolve web address — query rto_web_addresses in rto-nrt-db
Fetch robots.txt — extract Sitemap: directives
Fetch and parse sitemap — handle both sitemap.xml and sitemap_index.xml; cap at 500 URLs per RTO
Classify URLs — pattern-match against known categories
Detect external platforms — hostname matching
Write to D1 — structured JSON in radar_crawl_results

URL classification categories¶

Category	Pattern matches
`courses`	`/courses/`, `/qualifications/`, `/training/`, `/cert-`, `/diploma-`, `/certificate-`
`enrol`	`/enrol`, `/apply`, `/register`, `/enquire`, `/enquiry`
`student`	`/student`, `/portal`, `/my-`, `/login`, `/lms`
`contact`	`/contact`, `/locations`, `/find-us`
`policies`	`/usi`, `/fees`, `/refund`, `/complaints`, `/privacy`, `/terms`
`about`	`/about`, `/team`, `/staff`, `/governance`
`news`	`/news`, `/blog`, `/events`

External platform detection¶

Detected via both sitemap hostname matching (crawl pipeline) and link destination resolution (enrichment pipeline).

Platform	Hostname pattern
Canvas LMS	`*.instructure.com`
Canvas Catalog	`*.catalog.instructure.com`
Moodle	`moodle.`, `.moodlecloud.com`, `.moodlesites.com`, `.mdl2.com`
Blackboard	`*.blackboard.com`
Blackboard Collaborate	`*.bbcollab.com`
D2L Brightspace	`*.brightspace.com`
Teachable	`*.teachable.com`
Thinkific	`*.thinkific.com`
LinkedIn Learning	`learning.linkedin.com`
Wisenet SMS	`*.wisenet.co`
aXcelerate	`*.axcelerate.com(.au)?`
VETtrak	`*.vettrak.com(.au)?`
JobReady	`*.jobready.com(.au)?`
RTO Manager	`*.rtomanager.(com\\|com.au\\|net)`
Cliniko	`*.cliniko.com`
Google Forms	`docs.google.com/forms`, `forms.gle`
Microsoft Forms	`forms.office.com`, `forms.microsoft.com`
SurveyMonkey	`*.surveymonkey.com`
JotForm	`*.jotform.com`
Typeform	`*.typeform.com`
Eventbrite	`*.eventbrite.com(.au)?`
SharePoint	`*.sharepoint.com`

Enrichment Pipeline¶

Overview¶

The enrichment pipeline is a local Puppeteer script (tools/radar-enrich-local.mjs) that visits every RTO's homepage with a full Chromium browser instance and captures every signal extractable from that single visit. It runs on Tim's Mac Mini M2 Pro, not on Cloudflare Workers.

Why local, not CF Workers¶

CF Browser Rendering has a 2-concurrent-session limit — bulk work is infeasible
Local Puppeteer runs 10+ parallel Chromium instances with no concurrency ceiling
Local execution keeps the CF Browser Rendering budget free for ad-hoc per-RTO refreshes
The previous corpus screenshot ingest used this exact pattern successfully

Three lifecycle modes¶

Mode	Tool	Trigger	Cadence	Volume
Bulk ingest	Local Puppeteer (`tools/radar-enrich-local.mjs`)	Manual	On demand	All ~12,500 RTOs
Monthly refresh	Same local script	Manual, 1st of month	Monthly	All RTOs
Ad-hoc single	CF Browser Rendering via `radar-crawl` worker `/single` endpoint	"Trigger crawl" button in RadarTab UI	On demand	1 RTO

v0.3 reality note. The monthly bulk refresh never ran a second time — the Gen-1 corpus layer is frozen at the 2–3 April 2026 bulk pass (RADAR-SUBSTRATE-RECON-01). The freshness model in Product Architecture supersedes the monthly-refresh aspiration: ambient trickle + crawl-on-onboarding + subscriber watchlist, with bulk passes available as a tool, not a cadence. The three modes below remain accurate as mechanisms.

What the enrichment captures per RTO (single visit)¶

Screenshots (two per RTO):

Viewport	Size	R2 key
Desktop	1280×800, JPEG q75	`radar-screenshots/{rto_code}/homepage.jpg`
Mobile	375×812, `isMobile: true`, `deviceScaleFactor: 2`, JPEG q75	`radar-screenshots/{rto_code}/homepage-mobile.jpg`

Both captured on the same page load — desktop first, then resize to mobile with 500ms reflow wait.

SSL cert details — from the TLS handshake Chromium performs. Captured even on cert errors (--ignore-certificate-errors flag). An expired cert is itself a high-value signal.

HTTP response headers — full headers stored as JSON in enrichment_data.response_headers. Server, X-Powered-By, HSTS promoted to queryable fields.

Layer 3 fingerprints — all meta tags, OG data, generator, viewport, canonical URL, copyright year from the rendered DOM via page.evaluate().

Analytics/pixel detection — dual HTML regex + network request interception. IDs captured where possible (GTM-, G-, AW-*).

Compliance signals — RTO code visibility, CRICOS code, USI mentions, privacy/complaints/accessibility links, government program mentions. All regex-based, mechanical, no inference.

Third-party domain classification — every outbound network request logged, deduplicated by hostname, classified as vendor or infrastructure (see below).

Link destination resolution — homepage links in categories enrol, student, contact, plus all external links, resolved via HEAD request (GET fallback on 405). Max 20 per RTO, 10s timeout. Platform detection on final URL.

HTML archive — full rendered DOM, gzipped, stored in R2 at radar-html/{rto_code}/{timestamp}.html.gz. Enables future re-mining without re-crawling.

Favicon — fetched from detected <link rel="icon"> or fallback /favicon.ico. Stored at radar-favicons/{rto_code}/favicon.{ext}.

Performance — load time to domcontentloaded (ms), request count, page weight estimate.

Per-field partial failure¶

Each capture operation is wrapped in try/catch. If one fails (e.g. favicon 404), the rest of the RTO's enrichment continues. Success is tracked per-field in enrichment_success_flags:

{
  "screenshot_desktop": 1,
  "screenshot_mobile": 1,
  "html_archived": 1,
  "ssl_captured": 1,
  "headers_captured": 1,
  "metadata_extracted": 1,
  "analytics_detected": 1,
  "compliance_parsed": 1,
  "third_party_captured": 1,
  "performance_captured": 1,
  "link_destinations_resolved": 1,
  "favicon_captured": 0
}

Analysts can query "RTOs where link_destinations failed" and re-run just that capture without re-doing the full visit.

Third-Party Domain Classification¶

Every outbound network request during a page load is logged. After navigation completes, each unique hostname is classified as vendor (real service/product relationship) or infrastructure (CDN/font/analytics plumbing).

Classification rules¶

Check the hostname against the vendor patterns list (regex, takes precedence)
Check the registrable domain against the infrastructure set
Unknown domains default to vendor (conservative — better to over-surface than silently filter)

First-party filtering¶

Requests to the RTO's own domain or subdomains are excluded. Filtering uses the final URL's registrable domain (after redirects), not the stored web_address — handles domain migrations like www.cit.act.edu.au → cit.edu.au.

Vendor patterns (62 patterns)¶

Organised by category. Full list maintained in tools/radar-enrich-local.mjs. Key categories:

LMS: Canvas, Moodle (Cloud/Sites/mdl2), Blackboard, Brightspace, Teachable, Thinkific, LinkedIn Learning
SMS/RTO: Wisenet, aXcelerate, VETtrak, JobReady, RTO Manager, Cliniko
Form/Survey: Typeform, JotForm, SurveyMonkey, Microsoft Forms
Chat/Support: Intercom, Tawk.to, Zendesk, Freshdesk, Drift, Crisp
Consent management: Cookiebot, OneTrust, CookieLaw, Osano
Analytics: Hotjar, Cloudflare Web Analytics, HubSpot, Segment, Mixpanel, Amplitude
Booking: Calendly, Acuity, TryBooking, Humanitix, Eventbrite
Maps: Google Maps (maps.googleapis.com, maps.google.com)
reCAPTCHA: recaptcha.net, google.com/recaptcha
Video conferencing: Zoom
Payment: Stripe, PayPal, Braintree, Square, Westpac PayWay, eWAY, SecurePay
Email marketing: Mailchimp
CRM: Salesforce, Pardot, Zoho, Pipedrive
Microsoft: SharePoint

Infrastructure set (33 domains)¶

CDN/static assets (gstatic, googleapis, cloudflare, jsdelivr, bootstrapcdn, fontawesome, typekit), ad/tracking plumbing (googletagmanager, google-analytics, doubleclick, googlesyndication, facebook.net), social embed loaders (platform.twitter, platform.linkedin), WordPress infrastructure (wp.com, s.w.org), video embeds (youtube, vimeo).

Note: google.com is NOT in the infrastructure set — it's too broad. Specific Google services (Maps, reCAPTCHA) are handled by vendor patterns. Unclassified google.com requests fall through to vendor-unknown.

Output¶

Two arrays in enrichment_data:

third_party_domains — vendor relationships: ["static.hotjar.com (Hotjar)", "js-ap1.hubspot.com (HubSpot)", "unknown-vendor.com"]
infrastructure_domains — plumbing: ["www.googletagmanager.com", "connect.facebook.net", "fonts.gstatic.com"]

Named vendors include the detected name in parentheses. Unknown vendors show hostname only.

Link Destination Resolution¶

Homepage links matching specific categories are resolved to their final URL via HTTP HEAD request, with platform detection on the resolved hostname.

Categories resolved¶

enrol — links containing "enrol", "apply", "register", "enquire"
student — links containing "student", "portal", "login", "lms", "my-"
contact — links containing "contact", "location", "find us"
external — any link to a hostname different from the RTO's own domain

Resolution method¶

Extract matching <a> tags from rendered DOM (max 30 pre-filtered)
Take first 20 links (hard cap)
For each: HTTP HEAD request, follow redirects, 10s timeout
If HEAD returns 405: retry with GET
Capture final URL after redirects
Pattern-match final hostname against platform detection list

Output¶

{
  "link_text": "Student Login",
  "href": "/student-portal",
  "final_url": "https://myschool.canvas.instructure.com/login",
  "category": "student",
  "platform_detected": "Canvas LMS"
}

Platform detections from link resolution promote to lms_platform_detected, sms_platform_detected, and third_party_enrolment_system columns on radar_crawl_results.

Provenance¶

Every enrichment run records an observed_via map in enrichment_data showing which detection strategy produced each signal:

{
  "ssl_issuer": "tls_handshake",
  "page_title": "dom_query",
  "generator": "meta_tag",
  "ga4": "html_regex+network_request",
  "rto_code_on_homepage": "html_text_regex",
  "link_destinations": "href_head_resolution",
  "third_party_domains": "network_request_listener"
}

Purpose: when a signal returns surprising coverage (e.g. Meta Pixel at 5%), the provenance map tells you which detection strategy was used, enabling targeted improvement without re-crawling.

Crawl Results Schema (complete)¶

CREATE TABLE radar_crawl_results (
  -- Core (from sitemap crawl)
  rto_code              TEXT PRIMARY KEY,
  crawled_at            TEXT NOT NULL,
  web_address           TEXT,
  has_sitemap           INTEGER DEFAULT 0,
  sitemap_url           TEXT,
  page_count            INTEGER DEFAULT 0,
  last_modified         TEXT,
  screenshot_r2_key     TEXT,           -- desktop: {rto_code}/homepage.jpg
  screenshot_failed     INTEGER DEFAULT 0,
  no_web_presence       INTEGER DEFAULT 0,
  classified_urls       TEXT,           -- JSON
  subdomains            TEXT,           -- JSON array
  external_platforms    TEXT,           -- JSON array
  raw_sitemap_sample    TEXT,           -- JSON array, first 50 URLs
  crawl_status          TEXT DEFAULT 'pending',
  crawl_error           TEXT,

  -- Enrichment: redirect (v0.2)
  final_url             TEXT,
  had_https_redirect    INTEGER DEFAULT 0,

  -- Enrichment: SSL (v0.2)
  ssl_issuer            TEXT,
  ssl_valid_to          TEXT,
  ssl_days_remaining    INTEGER,
  ssl_cert_error        TEXT,

  -- Enrichment: fingerprint (v0.2)
  page_title            TEXT,
  generator             TEXT,
  mobile_responsive     INTEGER DEFAULT 0,

  -- Enrichment: compliance (v0.2)
  rto_code_on_homepage          INTEGER DEFAULT 0,
  cricos_code_on_homepage       INTEGER DEFAULT 0,
  cricos_code_value             TEXT,
  complaints_link_present       INTEGER DEFAULT 0,
  privacy_policy_link_present   INTEGER DEFAULT 0,
  accessibility_statement_present INTEGER DEFAULT 0,

  -- Enrichment: platform detection (v0.2)
  lms_platform_detected         TEXT,
  sms_platform_detected         TEXT,
  third_party_enrolment_system  INTEGER DEFAULT 0,

  -- Enrichment: performance (v0.2)
  load_time_ms          INTEGER,

  -- Enrichment: artifacts (v0.2)
  screenshot_mobile_r2_key TEXT,
  favicon_r2_key        TEXT,
  html_archive_r2_key   TEXT,

  -- Enrichment: tracking (v0.2)
  enrichment_completed_at TEXT,
  enrichment_success_flags TEXT,  -- JSON per-field success map
  enrichment_data       TEXT     -- JSON blob (see structure below)
);

`enrichment_data` JSON structure¶

{
  "ssl": {
    "subject": "cit.edu.au",
    "issuer": "E8",
    "valid_from": "2026-03-25T01:34:10.000Z",
    "valid_to": "2026-06-23T01:34:09.000Z",
    "protocol": "TLS 1.3",
    "san_list_note": "Not available via Puppeteer SecurityDetails API"
  },
  "response_headers": {
    "server": "cloudflare",
    "content-type": "text/html; charset=utf-8",
    "strict-transport-security": "max-age=31536000"
  },
  "meta": {
    "page_title": "Home : Canberra Institute of Technology",
    "meta_description": "...",
    "og_title": "...",
    "og_site_name": "CIT",
    "generator": null,
    "viewport_meta": "width=device-width, initial-scale=1",
    "mobile_responsive": true,
    "copyright_year": "2026",
    "canonical_url": "https://cit.edu.au/",
    "favicon_url": "/favicon.ico"
  },
  "analytics": {
    "gtm": true, "gtm_id": "GTM-XXXX",
    "ga4": true, "ga4_id": "G-XXXX",
    "meta_pixel": false,
    "hotjar": true,
    "intercom": false
  },
  "compliance": {
    "rto_code_present": true,
    "cricos_code_present": true,
    "cricos_code_value": "00001K",
    "usi_mentioned": true,
    "privacy_policy_link_present": true,
    "privacy_policy_url": "https://cit.edu.au/policies/privacy_policy",
    "complaints_link_present": true,
    "complaints_link_url": "https://cit.edu.au/about/student-and-community-member-complaints",
    "accessibility_statement_present": false,
    "accessibility_url": null,
    "government_program_mentions": ["Free TAFE", "Fee-Free TAFE"]
  },
  "third_party_domains": [
    "static.hotjar.com (Hotjar)",
    "script.hotjar.com (Hotjar)",
    "www.facebook.com"
  ],
  "infrastructure_domains": [
    "www.googletagmanager.com",
    "connect.facebook.net",
    "www.google-analytics.com"
  ],
  "link_destinations": [
    {
      "link_text": "How to apply",
      "href": "https://cit.edu.au/study/apply",
      "final_url": "https://cit.edu.au/study/apply",
      "category": "enrol",
      "platform_detected": null
    }
  ],
  "performance": {
    "page_weight_kb": 22413,
    "request_count": 205
  },
  "observed_via": {
    "ssl_issuer": "tls_handshake",
    "page_title": "dom_query",
    "ga4": "html_regex+network_request"
  }
}

IP Observations¶

Each RTO's domain is resolved to IP addresses, and each IP is enriched:

Field	Description
`hostname`	The domain resolved
`ip_address`	Resolved IP (A record)
`ip_version`	4 or 6
`ptr_record`	Reverse DNS
`asn` / `asn_org`	Autonomous System Number and organisation
`network_block`	CIDR block
`country_code` / `city`	Geolocation
`is_shared`	Whether the IP hosts multiple domains
`hosting_provider`	Derived provider name

UI Surfaces¶

Field Observer Map (Layer 0)¶

The default view when opening any RTO's Radar tab. Visible without clicking "Show detail."

Components: - Homepage screenshot — full-width, 240px tall, from R2 - React Flow topology map — horizontal flow graph showing the site structure: - Root node — homepage, domain name label - Section nodes — one per classified URL category with count badge (e.g. "Courses (31)") - Subdomain nodes — dashed blue border, separate column - External platform nodes — amber, rightmost column (e.g. "Canvas LMS") - Meta bar — domain, last crawled date, page count, sitemap status

Progressive disclosure: - Layer 0 (always visible): screenshot + topology map + meta - Layer 1 (tooltip on node hover): single most relevant signal - Layer 2 (node click): opens URL in new tab - Layer 3 ("Show detail" toggle): full intelligence panel

Graceful degradation: - No web presence: "No web address recorded for this RTO" + government signals still shown - No sitemap: screenshot only + "No sitemap detected" + simplified single-node map - No screenshot: map and signals still render; screenshot area hidden

Intelligence Panel (Layer 3)¶

Behind the "Show detail" toggle. Contains the full existing Radar content:

Dossier header — last crawled, crawl status, entity badge, risk flags, analyst notes, trigger crawl button
Classifier bar — visual summary: hosting tier, email maturity, SSL health, entity type, hiring signal, web age, org size, Wikipedia presence
Stack profile — technology table by surface
Screenshots — historical screenshot grid
Seek job listings — active postings (if detected)
IP observations — resolved IP table
Signal layers 1–5 — collapsible sections with per-signal management

Database Topology¶

Database	Binding	Role
`radar-db`	`RADAR_DB`	All Radar-specific data: dossiers, signals, stack profiles, IP observations, screenshots, crawl results
`rtopacks-db`	`NRT_DB` / `RTOPACKS_DB`	Read-only — RTO web addresses, org identity
`radar-screenshots` (R2)	`SCREENSHOTS` / `RADAR_SCREENSHOTS`	Homepage JPEGs (desktop + mobile), HTML archives, favicons

R2 key conventions: - radar-screenshots/{rto_code}/homepage.jpg — desktop screenshot - radar-screenshots/{rto_code}/homepage-mobile.jpg — mobile screenshot - radar-html/{rto_code}/{timestamp}.html.gz — archived rendered HTML (timestamped, accumulates) - radar-favicons/{rto_code}/favicon.{ext} — favicon (overwritten on refresh)

Hard separation rule: rtopacks-db is never written to by Radar. It is the sacred NRT corpus. Radar reads web addresses from it. All Radar intelligence is stored in radar-db.

Naming note (v0.3): database and binding names in this section predate the naming canon. Verify current canonical names against cloudflare-naming-canon.md and the resource inventory at brief-draft time (BRIEF-DRAFT-SUBSTRATE-VERIFICATION); RADAR-SUBSTRATE-RECON-01 (July 2026) is the most recent substrate census.

Field Observer — Standalone Product Positioning¶

Radar is an internal tool today. Field Observer is its external product name — the version that can be shown to clients and eventually extracted as a standalone product.

v0.3 note: the 2026-07-04 ruling assigns the own-org expression of this material to Marketer and the outward expression to Radar (see Product Architecture). Field Observer remains the name of the topology-map component both surfaces share. The positioning below predates the split and reads as the combined pitch: the disarming move and "how you appear from outside" items are Marketer's; the competitor and sector items are Radar's.

Tagline: "What's already visible — made useful."

The disarming move: When someone first sees their own RTO's Radar card, they feel a slight vertigo — "who gave you permission to know this about me?" The answer: nobody needed to. Everything here was already public. The value isn't in what we see — it's in the assembly.

What Field Observer shows a client: - How their organisation appears from the outside - What a prospective student, funding body, or regulator would see - What technology their competitors are running - Where their enrolment funnel actually goes - How current their site is

What Field Observer does not do: - No AI. No machine learning. No natural language processing. - No access to private data, authenticated systems, or internal networks. - No judgments, scores, or rankings. - A mechanical process. Attention, applied at scale.

Future Work¶

Items described here but not yet built. Briefs will be scoped separately. The customer-expression ladder (items 1–5) follows the ratified build order — each layer stands on the one below; RADAR-WORKSPACE-EXPRESSION-01 is the design brief that governs the ladder's surfaces.

The expression ladder (ratified order):

Register — search + RTO pages, the free navigation substrate. Public pages carry TGA-sourced public record only (see the public/private line).
CAPTURE-01 — TGA snapshot-and-diff. The feed's engine and the only source of temporal intelligence on the register side (swagger audit C5). Promoted from someday-necessary to the feed's build dependency.
Sector feed — CAPTURE-01's output rendered. Headlines included; tap-through depth carries held-depth markers.
Watchlist — subscriber-nominated RTOs join the fresh-crawl rotation.
Dossier depth — full intelligence surfaces: scope history, decision chains, footprint analysis, the comparative/duck-and-shark reads (absorbs v0.2's "Comparative view").

Substrate work:

Append-only migration — radar_crawl_results to revision rows keyed (rto_code, crawled_at); screenshots to timestamped R2 keys; overwrite paths retired in the same brief (MIGRATION-COMPLETION-DISCIPLINE). Precedes or accompanies the expression work.
Revision scrubber — the Street View timeline on the dossier (STUDIO-HISTORY-SCRUBBER-01 pattern). Absorbs v0.2's "Historical signal tracking".
Held-depth marker component — the entitlements seam, built pre-entitlements with every tap open. (ENTITLEMENTS-MODEL-01 is the platform-level dependency, not Radar's to build.)
On-demand crawl path (Radar-CF, reframed) — CF Browser Rendering serving onboarding, watchlist, and feed-item refresh crawls. Replaces the migrated-bulk-burst framing.

Carried forward from v0.2:

RADAR-ENRICH-UI-01 — surface the enrichment fields in the admin UI: compliance signals bar, link destination panel, third-party vendor list, performance metrics, mobile screenshot toggle, HTML archive download.
Anomaly detection — automated flagging of signals that don't match expectations (active registration + expired SSL, recent ABN cancellation + live website, etc.). Sits near the feed.
RADAR-ENRICH-COMMERCIAL — WhoisXML API integration for comprehensive WHOIS/RDAP data at scale. Replaces the sparse 5% coverage on domain_registered/domain_expiry. EXT-API doc required before deploy.
Configurable crawl depth — per-RTO override for sitemap URL cap, screenshot frequency, etc.
Element-level sitemap analysis — parse individual course pages to extract qualification codes, mapping sitemap URLs to TGA scope data.

Superseded: v0.2's "Client-facing Radar card" (own-org mirror) — moved to Marketer per the 2026-07-04 split; specified under Marketer/WS-PRODUCT-01, not here.

RADAR-SPEC-01 v0.3 — RTOpacks internal specification — July 2026

RTOpacks Radar — Product Specification¶

Changelog¶

How to Use This Document¶

What Radar Is¶

Who uses it¶

What it is not¶

Product Architecture (ratified 2026-07-04)¶

Two tiles, one telescope¶

The four product layers¶

Marketer as the demo¶

Commercial philosophy — held-depth markers, no walls¶

Full depth first — entitlements deferred¶

Data canonicity ruling¶

Append-only substrate law + the revision scrubber¶

Freshness model¶

The public/private line¶

Adjacent finding, filed for the signal roadmap¶

The Dossier Model¶

Dossier fields¶

Signal Layers¶

Layer 1 — Government Reality¶

Layer 2 — Public Presence¶

Layer 3 — Technical Fingerprint¶

Fingerprint signals¶

Analytics and pixel detection¶

Compliance signals¶

Layer 4 — Integration Signals¶

Layer 5 — Human Signals¶

Confidence Tiers¶

Stack Profile¶

Surfaces¶

Crawl Pipeline¶

Data flow¶

URL classification categories¶

External platform detection¶

Enrichment Pipeline¶

Overview¶

Why local, not CF Workers¶

Three lifecycle modes¶

What the enrichment captures per RTO (single visit)¶

Per-field partial failure¶

Third-Party Domain Classification¶

Classification rules¶

First-party filtering¶

Vendor patterns (62 patterns)¶

Infrastructure set (33 domains)¶

Output¶

Link Destination Resolution¶

Categories resolved¶

Resolution method¶

Output¶

Provenance¶

Crawl Results Schema (complete)¶

enrichment_data JSON structure¶

IP Observations¶

UI Surfaces¶

Field Observer Map (Layer 0)¶

Intelligence Panel (Layer 3)¶

Database Topology¶

Field Observer — Standalone Product Positioning¶

Future Work¶

`enrichment_data` JSON structure¶