11 — Integrations & their economics

Agents are only as effective as the data they can actually reach. A plan made from out-of-date or invented data is not based on evidence, so every third-party integration is listed here with its access mode, cost model, actual quota limits, the stage that uses it, and what happens when it fails. Two hard rules: (1) hypothesis cards must cite their data sources from this catalog (doc 10's "why this rank" expander shows them); an agent that couldn't reach a source says so in its contract report — it never guesses (doc 05). (2) Every metered integration has a budget line on NoticeOS's own ledger and fails closed as described in doc 06.

The catalog

Integration	Provides	Consumed by	Access	Cost model	Quota / limits in practice	On failure
GSC API	live collector: daily Google-search clicks, impressions, CTR, average position; daily archive: ten report families (doc 02)	Sense (S2), Attribute	service account; Full-user grant per property; read-only API scope	Free	live total query runs 96/day/site; the archive adds 40 base requests/day/property (ten report families × four dates), search-appearance expansion, and pagination. Search Analytics exposes at most 50k rows/day/search type and does not guarantee every row, so an archive is analysis-grade evidence, not a complete log	retain last good snapshot/archive, record the failed report manifest, suppress dependent flags
GA4 Data API	live collector: daily active users, sessions, page views, events; on-demand display read: exact distinct users in trailing 30- and 5-minute windows, users per minute over the last 30, plus today-by-hour against the same weekday last week; daily archive: eight report families plus an exact rolling 28-day event-user aggregate (doc 02)	Sense, Attribute	service account; Viewer grant per property; read-only API scope	Free tier (separate token quotas per property and request category)	daily-series query runs 96/day/property; open Tower displays share two Realtime requests/property per minute (both minute ranges, and the per-minute rows of the Wall's per-minute visitor chart), and one bounded Core hourly request/property per 15 minutes for today plus seven days ago; a display whose cache was emptied (a deploy, eviction, a new day or clock) reads at once instead of waiting out that cadence, at most one extra read per window; hidden tabs stop polling, and rejected requests enter a shared cooldown; archive adds 33 base requests/day/property (eight daily families × four revision dates plus one rolling aggregate), plus pagination, and caps each report at 250k rows. Data API is aggregate/modelled reporting, not raw event/session export. Event parameters (customEvent:*) return data only after an operator registers them as custom dimensions, and are never backfilled. Every GA4 collection sends returnPropertyQuota, so token spend is measured rather than inferred from request counts	retain last good daily/display snapshot and archive; record a failed durable report manifest; never turn a provider failure or a future hour into zero; today remains visibly incomplete; an unregistered custom dimension is its own manifest state (ga4_custom_dimension_unregistered), never an empty success; scheduled collection records an exhausted token budget as ga4_quota_exhausted; display reads return a safe quota category and next permitted attempt, and a bucket under 20% remaining raises ga4-quota-pressure before the failures start
Bing Webmaster Tools API	live: site-level daily clicks/impressions; archive: six report families (doc 02). The separate AI Performance UI report is not exposed by the documented API	Sense (S2 complement), Attribute	user-level API key on one central BWT account; sites verified per property (GSC import supported)	Free	one live request/site/day plus one shared site-list request and four archive requests/verified site/day — six on the day the two weekly families come due. Microsoft publishes no read-quota figure; query/page reports update weekly, are collected weekly, and can lag behind the current date	retain last good live snapshot; record each failed archive family independently; never fill an absent trailing day with zero or relabel undifferentiated impressions as AI citations
MS Clarity data-export API	session behavior aggregates (rage/dead clicks, quickbacks, script errors, scroll)	Sense (UX signals), hypothesis grounding	API token per project	Free, but 10 calls/project/DAY hard cap; trailing 72h only; ≤3 dims; 1,000 rows, no pagination	one call returns ALL metric blocks per dimension split, so URL + Device = full read in 2 calls; ALWAYS snapshot to store and analyze from disk; counts only — element detail needs dashboard replays; clarity.ms is adblock-DNS-listed (undercounts ~15–25%, rankings stay valid; can also block NoticeOS's own probes)	optional data source — degrade without an error, flag staleness
PostHog query API (below)	product behaviour: daily visits, event taxonomy, errors, rage clicks, real-visitor page speed (LCP/INP/CLS/FCP p75), declared funnels	Sense (UX and product signals), Decide (hypothesis grounding)	one read-only personal API key for the account (Project, Insight and Query read), connected on /integrations; region, projects and saved funnels discovered	Free per query	2,400 requests/hour, 240/minute, 3 concurrent queries, a 10-second execution cap and an hourly bytes-read budget per project; six bounded, server-aggregated queries per asset per day, run one at a time	keep the last archive, mark the source degraded, record the rejected or skipped family; a 429 stops the run; missing rows are not zeros
Chrome UX Report API (below)	page speed by origin: the 75th percentile of LCP, INP and CLS that real Chrome users saw over the trailing 28 days, for phones and desktops	Sense (S5 site checks)	one Google Cloud API key with the Chrome UX Report API enabled, connected on /integrations	Free	150 queries/minute per Google Cloud project; two queries per site per day, four when the bare domain has no data and its www. host is asked	keep the last reading; a site with too little Chrome traffic is recorded as no data, never as zero; a refused key stops the run
DataForSEO	top ranked keywords with demand/difficulty/CPC/SERP features, backlink stock + 90-day new/lost movement, Google/ChatGPT mention metrics, and live result pages for an operator-chosen head-term panel	Sense (S1/S1b/S3/S4), Decide (opportunity sizing, the AI-Overview gate)	one shared API login/password; launched domains are discovered from Postgres; tracked terms from config/serp-panel.json	Metered: roughly $0.26–$0.29 per property per weekly run, plus ~$0.004 per tracked panel term per device; each report records exact provider cost	five bounded calls/week/launched property plus one per tracked query per device; the site's saved market (United States · English by default), phone and desktop; 200-row ranked-keyword cap, a 31-term panel ceiling and a 20-result panel depth. A $0.25/report reserve makes the $25/month portfolio data cap fail closed before a call, and the provider card shows how much of that cap is left this month, counted from the costs the reports themselves recorded, plus the prepaid account credit with the date it was read — refreshed by ONE free appendix/user_data read at the end of every successful sweep, which is metered spend of zero and is never retried	retain the latest stored snapshot, mark the data source degraded, and never interpolate missing rankings, links, AI mentions, or AI-Overview state
Ahrefs	DR (free tier), site metrics (plan-gated)	Decide (sizing), Attribute (links)	MCP (existing)	Plan-dependent; many endpoints return 403 on the current plan	treat as best-effort enrichment only	fall back to DataForSEO backlinks
Cloudflare	R2 (raw archives), Analytics Engine (the default report counter for assets without a database), D1 asset-database backups (below); GraphQL analytics (edge traffic, crawler hits); AI Gateway (budget enforcement, fallbacks); health checks	everything; Act (deploys); doc 06 (caps)	API tokens, scoped per account	Infra ≈ free tier today; AI Gateway free (and its budget enforcement prevents overspend)	AE free tier covers the report counters' volume; GraphQL analytics rate-limited but generous	infra failure = incident, not degradation
GitHub App	PR create/read, checks, deploy webhooks, file contents	Act, Sense (deploy annotations)	App private key (the most sensitive credential — doc 06 rotation)	Free	5k req/hr/installation — ample	Act pauses; Sense keeps running
Model providers (via AI Gateway)	builder/verifier/judge inference	Act, Decide sweeps	keys behind the gateway only	Metered per token; the dominant OS cost	per-run/day/month caps, velocity breaker (doc 06)	fallback chain → pause Act
Ad networks	revenue reporting → shared financial reads	Sense (revenue collection)	Mediavine daily collector; CSV for other providers	Free reporting	provider estimates stay estimated until payment reconciliation	keep saved history; missing reports are not zero
Stripe (below)	subscription and product revenue: each completed UTC day's net charges and payments less refunds, per site → shared financial reads	Sense (revenue collection)	one restricted key with Balance and Charges read, connected on /integrations; the site that earns it names the Stripe account on its Data sources tab	Free	one balance read and one page of up to 100 balance transactions per request; a day is asked once, after 01:00 UTC; a first run reads the last 35 days, and a run that would need more than 100 pages saves nothing	keep saved days; a refused key stops the run; a day with no sale is recorded as 0, a day not yet asked is absent
Affiliate networks (CJ, Amazon)	commissions, EPC → ledger; CJ product feed (GraphQL ads.api.cj.com: live prices/sale prices/images/tracked deep links) + link-search promos → placement data	Sense (revenue collection), Act (placement refresh)	CJ: PAT + CID (developer portal); transaction reports filter by SID = per-placement conversion attribution	Free	a CJ feed is best read through a deterministic tool (doc 05): feed → committed catalog with accuracy checks (14-day price staleness ⇒ the card is marked degraded automatically; no promo without a real end date); Amazon reporting is degraded with a 60-day lag	wide intervals; book on payment
Uptime	up/down per site, with when it was last checked	Sense, incident detection	none — no account and no connect step: the ingest GETs each site's home page itself (runUptimeChecks, workers/ingest/src/hygiene.ts) on the hourly tick, reusing the nightly served-layer check and its hygiene_checks reading	Free	one request per site per hour; a site read in the last 30 minutes is skipped	a home page that does not respond twice in a row, 45 seconds apart, raises hygiene-home-unreachable at error (one failed try raises nothing and the row shows Up · 1 failed try); when NoticeOS itself cannot reach the network, no site is marked down (the egress gate) and the row shows Not checked
Discord webhooks	operator notifications — a new open error alert and a data source that stops working, and nothing else; the list is declared once in packages/contract (NOTIFIED_CONDITIONS) so the card and the sender always use the same list	ingest → operator	one webhook url, connected on /integrations (below)	Free	keep channels few — alert fatigue is a documented failure mode, which is why the list is two lines rather than a digest	fall back to email
Web search API	grounded scouting (categories 1–4, doc 13)	Scout sweeps	API key	Metered, cheap per query	budgeted per category; results cached in the store	the category skips that cycle
Deep-research runs	bounded multi-source investigations for high-stakes scout finds	Scout (categories 1/3/5)	via model gateway	The most expensive item — per-run cap mandatory	few runs/month; each run must name the find it investigates	skip; never substitute a shallow guess
Google Trends	demand timing/seasonality for scout finds	Scout, Decide sizing	public/unofficial API	Free	rate-limited; directional only	omit trend dimension
Wayback / End-of-Term archives	orphaned-demand mining	Scouting category 1	public APIs	Free	slow; send requests in small, spaced batches	the category pauses
Update/volatility trackers	Google update calendar + SERP volatility for doc-02 external annotations + doc-03 confound hygiene	Sense, Attribute, Scouting category 2	scrape/RSS	Free	secondary sources; corroborate before annotating	manual entry fallback
Marketplace feeds (Flippa-class)	acquisition monitoring against portfolio buy criteria	Scouting category 5 (quarterly)	RSS/alerts	Free tier	low frequency by design	the category stays inactive
HN (Algolia API)	news intake: tech/AI/industry signal, front-page + keyword queries	Scout news intake (collect tier)	public API	Free	generous; poll daily, not live	gap in news intake; record it
News/RSS aggregation	news intake: curated industry + AI-announcement feeds	Scout news intake (collect tier)	RSS	Free	select feeds strictly — every added feed adds noise	remove feeds that no longer publish
shadcn MCP + skills	component registry browse/install + guidance on composing components for Tower UI work (doc 14)	Act (UI builds)	MCP server	Free	registry-driven — agents assemble registry components instead of writing their own	build from in-repo components only

Mediavine revenue

Mediavine ships with NoticeOS as packages/mediavine; no separate installation, Go binary, browser automation, or external CLI dependency is required. Its private publisher GraphQL interface is isolated in that package. NoticeOS handles the credential store, scheduling, retries, settings, and financial reads. This is the pattern for bundled integrations: a provider client with a small typed interface, plus an adapter to the existing NoticeOS services. A general plugin loader is unnecessary for this integration.

1. On Integrations, Mediavine's Connect (or the site's Ad revenue row's Connect) opens the connect panel: your publisher email and password, then Connect. NoticeOS signs in and lists the account's sites before it saves the login; a rejected login is never stored. Credentials and access/refresh tokens use the existing encrypted credential store and key rotation, and the session and site list the sign-in produced are kept with them, so showing the list that follows makes no further request to Mediavine.
2. Each site is matched to the Mediavine site on its own domain and ticked; a Mediavine site in the account that matches no NoticeOS site is listed below them. Start collecting saves the site id on the site's ad revenue entry — the only field it writes — and runs the first sync at once: the current month so far (before 06:10 Pacific, through the day before yesterday).
3. A site with a saved Mediavine site id syncs daily until its Data sources row is set to Not using; mediavineSyncOn in packages/contract/src/mediavine.ts is that one rule, read by the sync, the row and the monitoring. Financials, Home, the TV dashboard, and asset revenue all read the same saved figures.

The scheduled target is 06:10 Pacific time, including daylight saving changes. Yesterday is requested explicitly, regardless of portal presets. The 06:00 availability observation comes from the operator; missing/null days are treated as incomplete and retried, never recorded as zero. Scheduler ticks at minutes 10, 30 and 50 merely check whether work is due. A successful day causes no more automatic provider calls that day. After downtime, collection catches up missing days in one bounded request (at most 366 days).

There are at most three automatic report attempts per day. Failures back off 20 then 40 minutes, with provider Retry-After taking precedence. Authentication failure stops automatic attempts until reconnection. Manual attempts are at least 15 minutes apart. Site discovery caches successful results for 15 minutes and also delays repeated failed probes. One durable, renewed lease serializes discovery, authentication, collection, and credential/settings changes.

Access tokens are reused and refreshed only when due or rejected; password login is reserved for a missing/rejected refresh session. The provider supplies access expiry; refresh-token lifetime is not assumed. Page views and status reads never contact Mediavine. Not using stops a site's collection; Disconnect deletes the encrypted credentials and session and leaves each site's saved Mediavine site id as it was. Both retain financial history. Connecting again lists those sites as already mapped, and Start resumes them. A site with stored revenue remains assigned to its original asset, preventing the same site's history from appearing twice.

Each changed day is filed in the revenue ledger (revenue_observations, the one table every revenue collector writes) under the source mediavine-journey, "Mediavine Journey ad revenue", in integer USD cents, naming the Mediavine site and the run that reported it. mediavine_runs keeps attempted ranges, outcome, and the portal summary versus the daily sum. Daily sums drive the shared financial_ledger view, the same way for every source. A reported summary discrepancy is shown, not distributed across dates. Missing or partial responses preserve the last complete data. Daily estimates replace an older Mediavine CSV estimate only once they cover its recorded window; unknown CSV coverage requires the full month. Reconciled payments, including their superseding lineage, take precedence. Original ledger rows are never rewritten.

Use Backfill or recheck dates to retrieve older reports or revisions. The small command-line interface uses the same NoticeOS collector and session:

pnpm mediavine status --asset example.com
pnpm mediavine sites
pnpm mediavine sync --asset example.com
pnpm mediavine sync --asset example.com --start 2026-08-01 --end 2026-08-31

All commands return JSON; --url selects a different NoticeOS origin. Credentials are configured in the UI. The earlier standalone mediavine-cli is a prototype, not a NoticeOS runtime dependency. The tables are included in the Postgres schema; schema changes follow the operator-only maintenance sequence in AGENTS.md.

Wall revenue outlook

The Wall replaces its daily-users headline with Projected ad revenue for the current month. It shows the projected total, estimated earnings per remaining day, and the projected change against the previous complete month. The headline's hover text includes reported earnings and their date. The existing daily traffic bar chart, comparisons, and date labels stay intact; only its headline shows the revenue projection and pace. Projections never become financial-ledger entries or reconciled revenue.

The model reads saved Mediavine reports and GA4 sessions; viewing the Wall does not contact either provider. Backfill about twelve weeks of revenue for initial calibration and a complete previous-month comparison. At least 21 matched traffic/revenue days and two observations of every weekday are needed. Each weekday uses its latest four traffic observations, weighted toward recent weeks. Earnings use the median revenue per session from up to seven days with similar traffic in the last eight weeks. Weekend behavior is estimated from the asset's observations, with no site-specific coefficients.

Under Ad revenue → Forecast holiday calendar, choose US, Canada, both, or weekday patterns only. Public holiday dates come from date-holidays (ISC code; CC BY-SA 3.0 calendar data). Holiday traffic is compared with nearby normal occurrences of the same weekday; a multiplier is estimated only after two distinct holidays have observations. It can represent an increase or a decrease. School vacations, private closures, and other unobserved events are not inferred.

When an upcoming holiday lacks enough evidence, current-month reports are missing, traffic is stale, or traffic is far outside the earnings history, the projection is hidden and the reason is displayed. The prior-month comparison also appears only once that entire month is reported. Reported earnings remain visible while a projection is unavailable. Forecasts assume recent patterns continue; they are estimates, not guarantees of payment.

How to connect one

Every provider is connected in the product, on the Tower's /integrations page; /health shows what each data source is producing. How the credential store works (encryption, store before environment, what a run records, the legacy environment fallback and Import from this machine) is written once, in the ingest README.

What Test connection costs is shown on the card. Most probes are a free read-only call. Three are not, and the card shows this before the button is pressed: Discord's posts a real message, and the OAuth app's and Clarity's make no call.

Provider (id)	Fields	Probe the Test connection button makes
Mediavine (mediavine)	publisher email and password	permitted-site discovery; session reuse and a 15-minute cache/cooldown are shared with collection
Google (google)	either a sign-in (GOOGLE_OAUTH_REFRESH_TOKEN, written by the flow — see Connecting Google) or GOOGLE_SIGNAL_ACCOUNTS, the account-label → {service_account_b64, properties} map	gets a read-only token, then Search Console sites.list and one GA4 properties/{id}/metadata read. Identical for both ways to connect; only the error message differs, because "add the robot as a Full user" and "check which account you signed in with" are different instructions. No day of data is requested and no reporting token is spent
Google OAuth app (google-oauth-app)	GOOGLE_OAUTH_CLIENT_ID, GOOGLE_OAUTH_CLIENT_SECRET	a format check only, on purpose: no free Google call can verify a client id and secret without sending a person through a consent screen, so the accurate result is looks right, now sign in. Not a card of its own — the Google card has fields for it
Bing Webmaster (bing-webmaster)	BING_WEBMASTER_API_KEY	GetUserSites — the same call the collector makes first
DataForSEO (dataforseo)	DATAFORSEO_LOGIN, DATAFORSEO_PASSWORD	appendix/user_data; free, and it returns the credit left before the metered collection stops — recorded with the time it was read so the card shows it with its date without any button press; the budget line beside it shows the monthly cap
Calendar feeds (calendar)	CALENDAR_FEEDS — name → secret ICS url	one bounded GET per feed, reported by label; the url is the credential and never appears in a test result
Chrome UX Report (crux)	CRUX_API_KEY	one records:queryRecord call on the first site the daily job reads (a reserved origin before any site exists, and in the connect panel). Both a record and "no data for this origin" accept the key
Stripe (stripe)	STRIPE_API_KEY, a restricted key	one GET /v1/balance: no money moves and nothing is written. A key without the Balance read grant is refused like a wrong key
Discord (discord)	DISCORD_WEBHOOK_URL — the whole https://discord.com/api/webhooks/… address	posts one labelled message to the channel, and the card shows this before the button is pressed. Discord does offer a read of the webhook object, but it would test the wrong thing: the catalog row above defines this data source as live when NoticeOS can deliver a notification — "not merely that a webhook URL exists" — and a webhook whose channel the operator lost still responds to a read. The url is the credential and never appears in a test result, and Discord's own error body is never shown back

What the notification channel carries

• Two conditions. NOTIFIED_CONDITIONS in packages/contract is both what the notifier sends and what the card lists: a new open error alert, and a data source that turns Failing (read through the same connectionState as the card's chip). Nothing else.
• Once per condition. The notification job runs hourly at :05. The notifications table records each condition only after a successful delivery, so a failed send is retried next tick and the Alerts row's notified mark means a message was delivered.
• A real delivery records a success on the credential (last_ok_at), as the Clarity export does.
• Without a notifications table, nothing is sent, and the card shows this; migrations are operator-only.
• Only the last 24 hours, so the first run does not resend older events, and at most ten lines per message with a count of the rest.

Clarity: the one per-asset credential

• One connection, one map. Clarity issues a token per project, so its credential is an asset-map field (asset id → that asset's token) on the provider's one connection. Per-asset changes the form (one input per asset) and the card (which assets have a key), not the store.
• The single-project CLARITY_PROJECT_API_TOKEN is one entry of that map, declared as legacyAssetBinding in packages/contract. The card counts it, Import from this machine moves it into the map, and the export records which slot it used. The map takes precedence wherever both name the asset, and the form has no input for the single token.
• Calls left today, per asset, counted from the archive_runs rows the export wrote (one per call, a failed call included) in UTC days. Reading it costs no provider call.
• Test connection makes no provider call. With ten calls a day and no free endpoint, the test reports which assets hold a token and leaves last_ok_at unchanged; only a successful 04:30 export sets it.

DataForSEO: the month's cap and the account's credit

• The bar shows the cap: month-to-date metered spend against monthly_caps.data_usd, the reserve that fails closed before a call. One bar for the portfolio, not one per asset; the cap is edited in /settings.
• The credit is shown beside it with the date it was read, from the free appendix/user_data read that Test connection makes and the weekly sweep repeats once after a successful run (never retried). It is stored as balance_usd and balance_seen_at on noticeos.integration_connections and shown as Account credit $18.72 · seen 2h ago. A reading older than fourteen days is shown in the warn colour; an unparseable one is dropped; a key rotation clears it.
• One sum. The card, /health, /settings, the Wall's pace and the collector's own cap gate all read loadMeteredDataSpend in packages/contract: each collected report's provider_cost_usd plus the research_log rows bought one question at a time, disjoint by actor.

Connecting Google

Google is the one provider with two ways to connect, and both stay valid: an install running on a service account does not have to switch. The service-account path requires an operator to create a service account (the "robot") in a cloud console, download a JSON key and grant that account access on every property one at a time; signing in does the same job with one consent screen. The card shows which one is in use.

The operator's steps are the Connect Google guide; how the round trip, the signed state, the expiry and the revoke are built is in the ingest README.

What it costs. Nothing at Google: the sign-in, Test connection and the property listing are free read calls, and no reporting token is spent until a collector runs (the catalog rows list those quotas). NoticeOS requests exactly two read-only scopes plus the operator's address: analytics.readonly, webmasters.readonly (deliberately notwebmasters, which can also verify and delete sites) and openid + email, so the card can show Connected as [email protected], which is the only way to notice a sign-in with the wrong Google account.

How it fails, and what the operator sees:

• A box unticked on the consent screen. A grant missing either read scope is rejected, not stored; otherwise a credential that reads Analytics but not Search Console would look connected and fail one collection a day later.
• redirect_uri_mismatch or a rejected http address. The redirect URI is derived from the address the browser is on, never configured, and Google rejects plain http except on loopback. A Tower reached by a LAN address links to its 127.0.0.1 address instead; a deployed Tower on https needs neither.
• The seven-day expiry. A consent screen left in Testing expires every refresh token seven days after it is granted, and Google publishes no API that reports whether a screen is published. The card shows a countdown (in the warn colour inside 14 days, so from the moment a seven-day grant is made) and has an it does not expire option for the operator to choose. Publishing the app is the real fix; the read-only scopes are not sensitive enough to need verification for a standalone install.
• A revoked or expired grant returns invalid_grant; the card and last_error show the one fixed sentence Google revoked this sign-in instead of reporting a network error, and the fix is to sign in again.
• Signed in, nothing mapped. A sign-in returns the GA4 properties and Search Console sites the account can access; the operator chooses which site each belongs to, on the site's Data sources tab. Until one is mapped the collector collects nothing and logs this once per pull instead of failing.
• Disconnect revokes the grant at Google first, best effort, then deletes it.

Grounding rules (why this doc exists)

• Tool access is via MCP/gateway from the agents' side — agents query live data during sweeps and builds (a title-change proposal cites the actual GSC CTR series it read; a reclamation card cites the actual backlink rows). The integration layer lets an agent base a task on evidence instead of writing plausible fiction.

• Freshness requirements per use: Decide may use day-old data; Attribute must respect the finalization windows (doc 02 distortions table); Act's verifier re-queries rather than trusting the card's snapshot when the decision depends on the data.

• Cost attribution: metered calls made for a change bill to that change-id; sweep calls bill to os-overhead. The monthly integration budget (target: ≤ low tens of dollars/mo for the data plane at current scale; inference is by far the largest cost) is a budget on NoticeOS's own ledger with the same fail-closed treatment as inference.

• Setup state is tracked in the Tower (asset detail → integrations tile): live | degraded | needs-setup | skipped | not-applicable per asset per integration (skipped = relevant but operator-declined for now, reason required; not-applicable = the catalog marks it as never applying to this asset). Release claims cite these states instead of personal judgement. Bing AI Performance data is imported from operator exports because the documented API does not expose that report.

  Clarity's two secret shapes. CLARITY_TOKENS (asset → project token) is the shape .dev.secrets.example.json shows and the only one that can hold tokens for the whole portfolio. CLARITY_PROJECT_API_TOKEN, a flat string bound to one site, is a documented fallback. The collector and scripts/creds-check.mjs apply the same precedence: the map takes precedence wherever both name an asset, so pnpm creds:check --lane clarity recognizes either shape (the probe runs only when requested explicitly because it uses 1 of the project's 10 daily calls). Migrating the secret to the map is a rename, not a rewrite.

  Connecting it. Clarity connects in the Integrations connect panel with one row per site: a token pasted on a site's row is saved at once (PUT /api/integrations/clarity/site-token, merged into CLARITY_TOKENS inside the ingest by workers/ingest/src/site-tokens.ts, so one paste never replaces another site's token), and the row shows the site's status. There is still no free call to verify a token, so the first export is an explicit Run now · 1 of 10 — the 04:30 job's own step (collectNow on the clarity job), for the sites that have a token and calls left, each spending one of that site's ten daily calls.

Credential naming — the ASSET_TOKEN convention

The scheme, in one sentence. Every asset holds ONE secret named ASSET_TOKEN — it gates that asset's metrics/self-report endpoint and authenticates that asset's pushes to the ingest Worker — and the ingest Worker holds ONE map, ASSET_TOKENS { "<asset-id>": "<token>" }, which it checks inbound pushes against and sends as the bearer on outbound pulls. The inbound pushes are the nightly report (POST /api/pulse) and the codebase report (POST /api/codebase-report, doc 27); the same token authenticates both. No other credential names exist between an asset and NoticeOS. A third-party provider credential (GA4, Bing, Clarity, DataForSEO) is a different thing entirely and keeps its own slot; this section covers only the tokens NoticeOS and its own properties use with each other.

One token per asset covering both directions is deliberate, not an oversight. The accepted tradeoff: a leaked ASSET_TOKEN lets the holder forge that asset's nightly and codebase reports as well as read its counts-grade metrics. That risk is accepted — the possible damage is limited to one property's counts and one property's ledger of daily numbers, both re-derivable from the property itself, and the flags a forged report could raise are visible to the operator on the Wall. Splitting the directions again to limit that damage would bring back exactly the second map this scheme deleted, with two secrets per asset that must be rotated together and that get out of sync without any error when they are not. If you are about to propose a separate pull credential, you are proposing PULL_TOKENS again — read this paragraph first.

Retired names. A search for one of these names, or an old note that mentions one, should lead here, not to a live credential:

Retired name	Was	Now
PULL_TOKENS (ingest Worker)	a second per-asset map holding the outbound scrape bearers	deleted — ASSET_TOKENS serves both directions
METRICS_SCRAPE_TOKEN (one asset)	the per-asset secret gating GET /api/internal/metrics	renamed to that asset's ASSET_TOKEN
ADMIN_TOKEN (another asset), and the copy of it the OS kept under its own name	the secret gating GET /api/admin/overview, shared with NoticeOS so a cron could read it	renamed to that asset's ASSET_TOKEN; the OS's copy is gone (scripts/pulse-relay.mjs reads that asset's ASSET_TOKENS entry for both hops)

Onboarding a new asset — three steps.

1. Create a strong ASSET_TOKEN on the asset's own worker (openssl rand -hex 32, then wrangler secret put ASSET_TOKEN), and gate its self-report endpoint on it.
2. Register the same value in the ingest Worker's one map: add the "<asset-id>": "<token>" entry to ASSET_TOKENS (.dev.secrets.json locally, wrangler secret put ASSET_TOKENS on a deployed ingest).
3. Prove it with a real call: pnpm creds:check --lane pull for the pull direction, and — if the asset pushes — one POST /api/pulse that returns 201. A checkmark alone does not prove it; the probe prints the counters the endpoint actually returned.

Rotation is the same three steps with new values, in that order: the asset's secret first, the map second, the probe third. Between steps 1 and 2 that asset's pull fails closed with asset-pull-failed, and the alert resolves automatically on the next successful pull — no other property is affected.

Worked example — a pull-only asset. The asset has no push cron of its own; the OS reads its envelope from GET /api/admin/overview on the nightly pull. scripts/pulse-relay.mjs (pnpm pulse:relay) does that same read when run manually and then forwards the envelope through POST /api/pulse — so it sends ASSET_TOKENS["example.org"] on both hops, outbound to the asset and inbound to ingest, and one entry satisfies both. The asset's ASSET_TOKEN is meanwhile pre-documented in its wrangler.jsonc as the future push credential, which is the whole point of the convention: the token an asset already has for being read is the token it will send when it starts pushing. Nothing is created, renamed, or added to the map on that day.

Where this is implemented. The mechanics are in workers/ingest/README.md — the Pulse entry of its Auth model and the pull's Auth — env.ASSET_TOKENS section — which link back here for the rationale instead of repeating it. scripts/creds-check.mjs names this section in its fix messages when a property rejects NoticeOS's bearer token.

Central Google signal collector

What the 15-minute GA4 and GSC collector gathers is defined in the signal contract; how it is built (the account map, token reuse, provisional days, append-only runs) is in the ingest README.

What it costs. One GA4 and one GSC request per property every 15 minutes, 96 a day per provider, whatever the date range: the 97-day window adds no requests. That is far below GSC's 1,200 queries/min/site and GA4's 200,000 tokens/day and 40,000/hour per property (most of these requests cost fewer than ten tokens). Every request also requests GA4's quota state, so spend is measured rather than inferred (GA4 quota).

How it fails. The Tower never calls Google; it reads the latest run for source health and the latest successful values for the charts, so a failed pull turns the source red while the chart keeps the last good values. The GSC headline figure is the latest completed day, so a partial current day cannot look like a collapse. A permission-denied attempt before a property's grant stays in the append-only history; the latest success determines health. A credential-label rename applies only from then on: rows under the old label stay as historical evidence.

GA4 realtime read path

The live visitor count is a display read, never a ledger or analysis dataset, and the one Google collection whose cost is set by a screen's poll rate rather than a schedule. One realtime request costs a property about 46–49 tokens; polled every 30 seconds by a Wall left on all day, that is roughly two thirds of the 200k-token daily realtime budget. So every open display shares one reading a minute (the two realtime requests behind it, both minute windows and the Wall's per-minute visitor chart), the today-by-hour Core read is refreshed every 15 minutes, hidden tabs stop polling, and rejected requests enter a shared cooldown. Any future cadence or multi-display expansion must be justified against the token use GA4 reports back, not a request count. How it is built is the ingest README.

How it fails. A valid empty report is zero; a failed or malformed one is an explicit unavailable state with null values, never a false zero. The Tower keeps the last good snapshot during a failed poll and labels it Reconnecting.

Daily provider analysis archive

Which GSC, GA4 and Bing Webmaster report families the 12:15 UTC archive keeps, and what each may be used to conclude, is the signal contract; how the archive job is built (revision window, probe mode, weekly families, content-hash dedupe) is the ingest README; the commands that collect and read it back are in scripts/README.md; what a property repo may conclude from the panel files is doc 20.

What it costs. Every call is free; the limits are request and row counts:

• GSC: 40 base requests a day per property (ten families × four revision dates), plus the search-appearance expansion and pagination. Discover runs in probe mode, so a property Google never shows in Discover costs one request a day rather than four.
• GA4: 33 base requests a day per property (eight daily families × four dates, plus one rolling 28-day aggregate), plus pagination; js-errors runs in probe mode too, so a property whose pages never throw costs one request a day for it. The archive keeps each collection's quota beside the pages, never inside the hashed bytes, and a bucket under 20% raises ga4-quota-pressure before the failures start.
• Bing Webmaster: one shared site-list request, then four requests per verified property a day, six on the property's own seventh day, because the top-query and top-page reports are weekly snapshots.
• Ceilings: 8 MiB per provider response, 32 MiB per archive, 50k rows per GSC report/day and 250k per GA4 report/day. A capped report is stored as provider_truncated=1, never described as complete. The ceilings follow Google's Search Analytics extraction guidance and the GA4 Data API pagination contract.

One operator step. The GA4 js-errors family reads event parameters, which return data only after an operator registers them as custom dimensions in GA4 admin and are never backfilled. Registration is operator work (the analytics pipeline is operator-only per AGENTS.md), recorded in config/ga4-custom-dimensions.json; until then the family is skipped or records ga4_custom_dimension_unregistered (details).

How it fails. Each family fails on its own manifest row; the Tower combines the Bing data source's families into one status, so a weekly family that was not due never turns it red. A refresh of the local panel files makes zero provider calls: it materializes archives the cron jobs already paid for.

Microsoft Clarity export

workers/ingest/src/clarity-dumps.ts runs at 04:30 UTC, last in the nightly chain because it depends on nothing the earlier slots write. It is the portfolio's hardest-capped provider — 10 calls per project per DAY, trailing 72 hours only, ≤3 dimensions, 1,000 rows and no pagination — and every design choice follows from that cap:

• One call per property per day, one dimension (URL), archived verbatim to R2 under one archive_runs row (integration = 'clarity', report = 'url-3d'). Analysis re-reads the stored object; re-reading must never cost one of the ten. A second dimension split is a second call and therefore a separate budget decision, not a free addition.
• A property with no token is skipped — no call, no manifest row, no error. The run log counts them as skippedNoToken so an all-skipped run cannot be mistaken for a clean run. This is the same rule the SERP panel applies to a property with no configured queries: absence of config is not a failed collection.
• Failures split by the operator action they need: clarity_token_rejected (401/403), clarity_daily_cap_reached (402/429 — the ten are spent), clarity_invalid_response (the call was spent and the response held nothing recognizable — a failed observation, never a property with no behavior), clarity_http_<status> otherwise.
• A block on the 1,000-row cap is provider_truncated. There is no pagination past it, so the number is a floor.

The flattened family is clarity-url-3d.csv, written long — one row per (metric, URL) — because the blocks genuinely do not share a schema: Traffic counts sessions and bots, EngagementTime reports seconds, ScrollDepth a single average. Traffic also has a null URL for its unattributed aggregate, which stays empty rather than becoming a URL of "". Sessions are sampled and clarity.ms is adblock-DNS-listed (undercounts ~15–25%), so these are behavior rankings, never population counts — the caveat is recorded in summary.json.

The manifests determine the data source's health. The Tower reads each asset's newest clarity manifest the way it reads DataForSEO's and PostHog's (apps/tower/worker/integration-evidence.ts): a success inside two days is Working, a failed export (a rejected token, the ten calls spent, an empty response) or one older than two days is degraded with the stored reason, and no manifest at all is not set up. That status sets the Clarity mark on the source strip (Wall, Home, asset header), the integrations matrix and the asset's Data sources tab, so it matches the ingest's own health read on /health. A data source declared skipped or not-applicable keeps that setting. An asset that moved to another behavior source should be declared skipped and lose its token, so the 04:30 export stops spending a call on it; the Tower does not infer that from dates.

Chrome UX Report

Daily page-speed reading. Collector: workers/ingest/src/crux.ts.

The Chrome UX Report (CrUX) API returns, for one origin, how fast pages loaded and responded for real Chrome users over the trailing 28 days. The key names the Google Cloud project whose free quota the calls count against; the data is public, so one key serves every site and connecting it lists no sites. Create the key on the Google Cloud console's Credentials page, enable the Chrome UX Report API on the same project, and paste the key into the Chrome UX Report (page speed) card on /integrations.

What runs, and when. The Page speed job runs daily at 04:15 UTC (15 4 * * *). For each site the hygiene checks read (a domain, not retired, not the System) and whose Data sources row does not say Not using, it asks CrUX for the site's origin twice, phones then desktops, and keeps the 75th percentile of Largest Contentful Paint (ms), Interaction to Next Paint (ms) and Cumulative Layout Shift. When CrUX has no data for the bare domain, the www. host is asked once. The run stores one reading per site per day in hygiene_checks (check crux-vitals): value_num is the phone LCP, or the desktop LCP where CrUX has no phone data, and detail holds both form factors and the dates CrUX aggregated. Page speed on the site's Data sources tab shows the newest reading.

What it does not do. No alert rule reads the readings yet. With no key connected the job records crux_not_connected and asks nothing. A key CrUX refuses (HTTP 400, 401 or 403) stops the run, stores nothing and marks the credential failed; a throttle or a server error fails only that site. When this machine cannot reach the network, the sites are named on the one os-egress-down alert, as for every other collector.

Stripe

Daily subscription revenue. Collector: workers/ingest/src/stripe.ts.

Stripe holds a site's subscription and product sales. NoticeOS reads them with one restricted key that can only read: create it on Stripe's API keys page with Read on Balance and on Charges, and paste it into the Stripe (subscription revenue) card on /integrations. Connecting checks the key with one balance read and lists no sites. Then, on the site's Data sources tab, the Subscription revenue row's mapping takes the account id (acct_…, on Stripe's account settings page). One key reads one account, so one site names it; while two sites name a Stripe account, neither is collected and both rows show the mapping failure.

What runs, and when. The Subscription revenue job runs at 20 minutes past every hour (20 * * * *). It asks Stripe only when a completed day is missing: a UTC day is complete from 01:00 UTC, the clock its revenue source row (stripe, "Stripe subscriptions", family subs) records. The first run reads the last 35 days; later runs read from the day after the newest saved day, at most 35 days back. Each run reads the account's balance (its one currency) and its balance transactions for those days, and for each day sums the net of charges and payments less refunds, in minor units. Fees, payouts and transfers are not revenue. Each day is one row in the revenue ledger (revenue_observations, source stripe, the account id as account_ref), a day with no sale included as 0; a day whose figure equals the saved one writes nothing. The month's days stand in for the month on the Money page and in each site's daily revenue, as Mediavine's do.

What it does not do. With no key connected the job records stripe_not_connected and asks nothing. A key Stripe refuses (HTTP 401 or 403) stops the run, saves nothing and marks the credential failed. An account whose balance holds more than one currency is not collected. The run's log line counts sites and names failures by code; it carries no amount and no key.

PostHog

Product analytics archive. Collector: workers/ingest/src/posthog-dumps.ts. Archive contract: packages/contract/src/posthog.ts.

Search data shows who arrives. PostHog data shows what they do once they are on the page: which steps of a journey they finish, which errors they hit, where they rage-click, and how fast the page is on their own device.

What runs, and when. A daily run at 12:30 UTC (30 12 * * *). It starts after the 12:15 Google/Bing archives and before the 13:10 local summary rebuild. For each launched asset that has both a key and a saved region and project, the run reads the project's timezone. It then runs seven queries, one at a time, each ending on the last complete day in that timezone:

Family (report)	Window	One row is	Bound
web-daily	trailing 28 days	one day: pageviews, unique people that day, sessions	28 rows
events	trailing 14 days	one event name: count, unique people, first and last day seen	top 500 by count
exceptions	trailing 14 days	one error type + message (first 200 characters): count, people, sessions, most in one session, whether any stack frame names a source file, top page, top browser	top 100 by count
rageclicks	trailing 14 days	one page + element (tag, text, name-or-id attribute): clicks, people, desktop/mobile/tablet clicks, people who viewed that page	top 100 by people
web-vitals	trailing 14 days	one page × device × OS: 75th-percentile LCP, INP, CLS, FCP and the measurement count	top 20 pages by measurements, all their segments, at most 300 rows
inp-targets	trailing 14 days	one page × device × OS × the element behind each page view's slowest interaction (its CSS selector, first 200 characters) × pointer or keyboard: page views, 75th-percentile INP and its three parts (input delay, processing, presentation delay)	the same top 20 pages, at most 300 rows by page views
funnels	trailing 7 days	one step of a declared funnel: people who reached it in order within the window	the declared steps

Window grain, not day grain. Unique people and percentiles cannot be added across days, so a 14-day total built from daily rows would be wrong. So events, exceptions, rageclicks, web-vitals, inp-targets and funnels are one row per item over the whole window. Only web-daily has one row per day, and its people is unique within that day. The archive body records its own window (window.start/window.end, inclusive, in the project's timezone), so a reader never has to assume one.

Where it is stored. Each family is saved as one R2 object per asset and window end, through the ordinary signal-dump path. Every attempt adds one report run (noticeos.archive_runs: integration = 'posthog', report = family, report_date = window end). The object is the standard envelope with one page. That page's response is the contract body, checked against its zod schema before anything is stored. The body's own collectedAt is left out of the content hash, so re-reading a settled window is recorded as unchanged.

Read-only, bounded, budget-aware. The collector calls exactly two endpoints:

• GET /api/projects/{id}/ reads the project's timezone.
• POST /api/projects/{id}/query/ sends one HogQLQuery.

Both use Authorization: Bearer <key>. Every query:

• aggregates inside PostHog;
• filters on timestamp to its window;
• ends in an explicit LIMIT of the bound plus one (the extra row is how truncation is detected; it is then dropped and the archive marked truncated);
• never uses OFFSET, which PostHog rejects for personal keys.

PostHog allows 3 concurrent queries per project and an hourly bytes-read budget. Families therefore run one after another, and X-PostHog-Query-Budget-Remaining-Bytes is read on every response. A 429 stops the whole run. The rejected family gets an error row: posthog_query_budget_exceeded for the budget, posthog_rate_limit for the request rate. Every family not yet requested is listed as skipped on the completion line, with no request made and no row written.

One run per asset at a time. Every run takes the asset's lease before its first PostHog request: the daily run, the runner's runScheduled and signals:collect alike. The lease is one integration_leases row, provider = 'posthog:<asset>', so no migration is involved. It is released when the run ends, however it ends. A run that crashed leaves it in place for at most POSTHOG_LEASE_MS (30 minutes), more than four times the longest possible run. A run that finds the asset's lease taken sends PostHog no request for it. On demand that returns 409 collection_in_flight, with when the other run started and when the lease expires. The daily run skips that asset with the reason in-flight and continues with the others.

Skipped, with the reason recorded. In each of these cases there is no call and no row:

• no key and no project;
• a project saved but no key (no-key);
• a key but no saved region or project (mapping-missing);
• a saved region or project that is not valid (mapping-invalid);
• no funnels declared (only the funnels family is skipped);
• another run already collecting that asset (in-flight, above).

Failures. Each failure's code shows what to fix:

• A key PostHog rejects (401 or 403) is posthog_access_denied. It is written for every family that was due and never crashes the run.
• A project that is not in that region is posthog_mapping_project_not_found.
• A response whose columns or values do not match the query is posthog_invalid_response.
• A response this machine could not save (a Postgres or R2 failure while archiving it) is local_store_failed, on every archive collection, not just PostHog. The Integrations page shows it as NoticeOS's fault, not the provider's. PostHog's key status does not count it, and the daily run never requests it again: the provider already responded.

An offline night. When a PostHog call gets no response, the collector first checks NoticeOS's egress gate. When this machine is the one offline, no row is written for the window and it is listed on the one os-egress-down alert instead. A window end listed on that alert, or whose latest attempt failed at the network (posthog_request_failed, posthog_timeout), is requested again by the next daily run: up to six a run per asset, oldest first. A rejected key (401/403) is not retried daily. Details: workers/ingest/README.md.

A temporary error from PostHog. A window for which PostHog returned a 5xx, a malformed response or a 429 is requested once more by the next daily run, within the same six and order. The same kind of response twice for one window is treated as final and stays on the Integrations page. A 429 still stops the whole run. The windows it did not request, on this asset and every asset after it, are requested by the next run too: the 429's own row records the stopped run, so no other record is written.

Configuration: everything the operator sets, and where.

Setting	Where	Notes
Key (POSTHOG_API_KEY: one personal API key for the account)	/integrations, the connect panel	Create the key under PostHog → account settings → Personal API keys, with read access to Project, Insight and Query. Connect sends it to the US and EU clouds at once and saves it only where it is accepted; that cloud is the region. An install that connected a key per site before (POSTHOG_KEYS: asset → key) keeps collecting with it — a site with its own key uses it, every other site the account key; the Tower no longer has a field for a per-site key. The env bindings of the same names are the legacy fallback.
host (us or eu) and projectId (digits)	Discovered: the connect panel lists the region's projects, matched to sites by the domain each project records (its app URLs, then its recording domains, then a name that is a domain), and Start saves them on the site's Data sources entry	Standard mapping fields, still editable on the Data sources row. No fallback: a guessed region or project would read another site's data.
funnels (list of { id, name, steps: [{ event, path? }] }, 2–10 steps each, at most 10 funnels)	Imported: the project's saved funnel insights, written by Start only where the site's entry holds none	Only funnels whose every step is a named event, optionally on one exact $pathname, are imported — the archive cannot count any other filter as PostHog does. Checked by posthogFunnelsRefusal, the same rule the store save runs.

When the panel calls PostHog. Only after Connect is pressed (two GET /api/projects/ reads, one per region, at once) and when the panel lists the account's sites (the project list, then each project's details and saved insights, at most 20 projects). Every read is bounded at 10 seconds; a rejection in both regions shows "PostHog rejected this key", anything else "No response from PostHog", with Try again (workers/ingest/src/posthog-account.ts).

Connection test. The Test button reads each mapped site's project settings once (the account key's project list when no site is mapped yet). No query runs, so none of the hourly budget is spent.

Running it on demand.

pnpm os:collect -- --asset example.com --families 'posthog-*' [--start 2026-09-08 --end 2026-09-22]

pnpm os:collect runs pnpm signals:collect inside the app container, where the ingest's door is; in a checkout that runs NoticeOS without Docker, run pnpm signals:collect with the same arguments.

Quote posthog-* in zsh, or write posthog. With --start/--end, every family uses that one inclusive window (at most 28 days), so a past manual reading can be reproduced. A run with a fixed window shares its report_date with the daily run that ends on the same day. Read window from the body. Without a fixed window, the run also requests the earlier windows still due for the asset, as the daily run does. The body lists them in retried and counts the ones left for the next run in retryNotAsked.

Event notes.

• First-party events reach PostHog by name only. A missing property is a privacy choice, not missing data.
• Events behind consent arrive only after consent, so they are a floor.
• Events whose $lib is the site's own worker (e.g. example-worker) are server-side and use different ids.
• people is uniq(person_id), PostHog's own unique-person count. A manual reading that counts distinct_id instead differs slightly wherever PostHog has merged a person across ids.
• Web-vitals percentiles use quantileExact, so a re-run gives the same result. PostHog's own dashboards use an approximate percentile.
• inp-targets reads the attribution PostHog's browser library keeps for INP by default ($web_vitals_INP_event.attribution: interactionTarget, interactionType, inputDelay, processingDuration, presentationDelay). An event from a library version that kept none has an empty element and empty parts: unknown, not a fast interaction.

Checklist, filled.

• Purpose: product behaviour.
• Consuming stages: Sense and Decide.
• Auth: one read-only personal API key for the account in the credential store (doc 06); an older per-site key map still collects.
• Cost: none per query. The only limit is PostHog's hourly budget.
• Quota and distortions: above, plus adblock undercount and consent floors.
• Degradation: keep the last archive, mark the source degraded, record the rejected family, and never fill a missing row with zero.
• Tower: collector-backed. The Data sources tab shows working or degraded from real runs.

Weekly DataForSEO search-intelligence archive

workers/ingest/src/dataforseo-dumps.ts runs Mondays at 12:45 UTC for each non-OS asset whose lifecycle is neither pre-launch nor retired. The canonical domain comes from Postgres. Each property gets five independently durable domain-driven report families, plus a sixth where an operator has chosen head terms to track:

• ranked-keywords: the top 200 current Google results in the site's saved market (United States · English by default), ordered by estimated search volume, with rank, URL, demand, CPC, difficulty, intent, SERP features, estimated visits/cost, rank-change fields, and AIO references;
• backlinks-summary and backlinks-new-lost: current authority/link stock plus DataForSEO's weekly-grouped 90-day movement series;
• llm-mentions-google and llm-mentions-chatgpt: platform totals and the leading source domains from the current target-metrics endpoint;
• serp-panel (doc 08 §S1b): one live result page in the site's saved market (United States · English by default) per tracked head term per device, with the asynchronous AI Overview requested, read to depth 20. The panel comes from config/serp-panel.json — the one place a tracked-query list is written down, capped at 31 per property (what the $0.25 per-report reserve buys at two devices). A property with no panel is skipped without an error: no call, no manifest row, no attempt, because absence of config is not a failed collection. The live/advanced endpoint takes one task per request and the device is a property of the task, so a panel is (terms × devices) calls stored in one archive under one manifest row — which is why the second device needs no migration, no second family and no second review task: device is stored in the archived request body and comes out as a CSV column. A term the provider accepts but returns no result for is archived with the others instead of the run being discarded, and a panel whose every term failed is an error, not an empty success. Every panel run is flagged truncated, because each result page continues past depth 20; the flag stays on the run as a fact and the collection counts as complete. A panel with unanswered terms is recorded as an incomplete report.

Responses use the same immutable gzip-JSON R2 envelope and append-only Postgres manifest as the daily provider archive. Every provider response retains its exact cost. The manifest also indexes that compact cost so the scheduler can stop before a call could cross the existing $25/month data cap; a conservative $0.25 reserve is required before every call. Missing credentials, budget exhaustion, provider errors, and partial runs are shown explicitly as red/degraded in the same property integration icon. A tracked panel costs ~$0.004 per term (~$0.08/week for a 20-term panel) and needs the same $0.25 reserve as every other family — the gate has no exception for panels, and a malformed panel fails that one property with config_invalid while every other property still collects.

The cron only starts a check; it does not mean the same snapshot is bought again. Every family declares a seven-day cadenceDays in the collector's machine-readable report registry, exported as DATAFORSEO_REPORT_CADENCE_DAYS. An automated cron or startup catch-up checks the newest success/unchanged manifest for each property and family, skips it until that cadence is due, logs the skipped family with its next due date, and writes no placeholder attempt. Thus a later successful sweep satisfies a missed Monday job even when the runner's job record shows that original invocation failed. A failed family remains due. The explicitly scoped operator command is the deliberate exception: it may force only the named property/families for a new baseline or repair, while the collection lock still prevents concurrent rebuying.

The analyzer reads locally retained weekly snapshots. Current executive facts use the newest snapshot; ranking-movement findings compare the newest two; backlink momentum uses the stored 90-day provider series. When a genuinely due response matches its prior archive, the collector appends an unchanged row recording that the provider was queried, and reuses the prior R2 object. Cadence skips keep the earlier manifest current and write no synthetic row. Analysis therefore never pays to retrieve historical DataForSEO data again. Provider search volume, difficulty, estimated traffic/cost, and AI search volume remain modelled prioritization inputs—not causal ROI, sessions, or unique people.

The compact property read model publishes up to 16 useful query rows from the latest ranked-keyword archive, including current organic position, demand, difficulty, page, AI Overview presence, and provider-returned AI-reference position. It is available on the first snapshot. A stored previous organic position and movement appear only for an exact normalized-query/page match in the preceding snapshot. Where the tracked panel covers one of those queries, the row also includes the live result page's AI Overview presence and whether it cited this property — three-state, matching by normalized query. The panel is additive: it contributes no rows of its own, and a query it does not cover (or one whose overview failed to load) stays unknown, which every rule downstream treats exactly as it treated an untracked query before the panel existed. The Tower merges this with Google/Bing rows by query for display while keeping the units separate: webmaster impressions are observed property exposure; DataForSEO search_volume is modelled demand in the site's saved market (United States · English by default).

A property that reaches the explicit 200-row keyword cap has that family marked truncated in the manifest rather than implying complete keyword coverage.

The analysis tool keeps missing rows unknown and outputs provider dimensions and metrics without inventing totals. Its row_grain column distinguishes the search-appearance discovery totals from the filtered page rows; those two grains must never be summed together.

This archive can support opportunity discovery, page-extension/deprecation reviews, and feature-behavior analysis, but it is not causal evidence by itself. Decisions still join it to deploy/change annotations, the product's nightly report, revenue/cost rows, and (where possible) holdouts/cohorts. GSC page/query exports can omit anonymized or low-volume rows, and GA4 Data API reports are aggregates, not raw event/session sequences. GA4 BigQuery export remains the later option for funnels, retention, and user/session paths.

Google's separate Search Generative AI report exposes AI Overview/AI Mode impressions in the Search Console UI, but the documented Search Analytics API has no corresponding report type. Do not label ordinary Web Search rows as AIO-specific. Until Google adds API access, known assistant referrals come from GA4, broad AI visibility comes from the weekly DataForSEO Google/ChatGPT mention snapshots, and exact query-level citation detail comes from the tracked serp-panel above — for the terms on a property's panel, and no others. Property-wide AIO/AI-Mode impression totals still require an operator-downloaded Search Console generative export.

Bing AI Performance boundary

Bing Webmaster Tools' public-preview AI Performance report adds citations, cited pages, grounding queries, citation-to-page mappings, time series, and preview intent/topic/citation-share comparisons across Microsoft Copilot, Bing AI summaries, and participating partner surfaces. The operator can export it as CSV/Excel from the Bing UI.

Those dimensions are absent from the documented IWebmasterApi method surface. Search Performance impressions can include a chat response, but the standard API does not identify that subset. NoticeOS therefore keeps AI Performance as a separate data source that the operator imports, and does not derive AI citations from ordinary BWT query or traffic rows.

The import. The parser is fixed to the formats of three real exports, not to an assumed format: workers/ingest/src/bing-ai-exports.ts reads exactly three header rows — AIPerformanceOverviewStats (daily Date/Citations/Cited Pages), AISearchQueriesReport (Grounding Query/Intent/Topic/Citations/ Citation Share), and AIPageStatsReport (Page/Citations) — and rejects anything else rather than mapping columns by position. UTF-8 BOM, fully-quoted fields, M/D/YYYY 12:00:00 AM dates, 12.34% share strings and digits-only counts are all part of the fixed format; a thousands separator causes a rejection, not a conversion.

The operator runs one command:

pnpm bing-ai:import ~/Downloads/example.com_AISearchQueriesReport_8_4_2026.csv

scripts/bing-ai-import.mjs reads the property and the export date from Bing's own filename (<site>_<Report>_<M_D_YYYY>.csv — the only place two of the three exports include a date at all), POSTs the bytes to the ingest's loopback endpoint (POST /api/bing-ai-export, operator-authed) and refreshes the property's panel dir. The archive uses the standard format — gzip JSON in private R2 with an append-only archive_runs row, integration bing-webmaster, reports ai-overview / ai-queries / ai-pages, data_state='provider-snapshot', report_date = the export date — and it holds the original file byte-identical beside the parse, so the file can always be parsed again. Content addressing makes a re-import unchanged rather than a second copy; a later export appends as the next dated snapshot. The panel contract (grain, and what an absent family means) is doc 20 §The bing-ai family. Dashboard scraping remains outside scope.

Central Bing Webmaster collector

workers/ingest/src/bing-signals.ts runs once on the existing 02:30 UTC nightly pull tick because Microsoft updates GetRankAndTrafficStats daily. BING_WEBMASTER_API_KEY is a user-level credential: one key lists every site verified under the operator's account. The Worker intersects that list with launched, non-OS portfolio assets, then calls the traffic endpoint once for each match.

The provider returns daily clicks and impressions without date-range request arguments. The Worker retains the actual returned dates inside a rolling 97-day retrieval horizon and appends them through the same change-only signal store used by Google. The Tower renders at most 90 of those dates and uses up to seven earlier dates only as calculation context. It does not fill missing dates or the dates after Bing's latest row: unreported provider dates are unknown, not zero. The Tower reads Google and Bing separately, aligns them by calendar date on one zero-based Web Search Clicks scale, colors Bing with the provider-identity blue token, and lets either line end on its own latest date. Both retain same-weekday-last-week references. Bing's retention reaches back well past four weeks; a shorter series means a new property, not a provider limit. A property moved to live with sense_only=1 participates in the launched-property collectors without enabling an Act loop.

Cloudflare D1 backups

Connect Cloudflare in Integrations with the account ID and an account-owned API token restricted to that account's D1 resources. Use Account → D1 → Edit; Cloudflare documents this scope for its D1 REST API setup. Do not grant DNS, Workers deployment or unrelated account permissions. The API reference does not publish a narrower accepted-permission list for export; a read-only token may not be able to export.

Connect and Check read the account's database list. That proves list access, not export permission. The token uses the existing encrypted workspace store; it is never returned to the browser. The account connection is separate from any Cloudflare credential used to host NoticeOS itself.

Choose each D1 database and its portfolio asset in the same connection panel, then press Back up selected. No asset is inferred from a database name. Token replacement preserves selections; changing accounts requires an explicit selection for the new account. Hosted calls recheck owner membership in both Workers; demo visitors cannot list provider databases, select targets, export, or retrieve private SQL.

An export polls the native API with the original continuation bookmark, then downloads SQL without sending the API token or following redirects. The current bounds are 100 selected databases, three minutes per export and 4 GiB per SQL artifact. SQL streams through 8 MiB multipart buffers with a SHA-256 hash into the existing private object binding, under workspace/account/database/run IDs. Receipts are stored in the workspace's provider-coordination row; a transaction checks the current connection, selection and run lease before publishing completion. An expired run cannot publish over its successor. A failure keeps the last complete backup; successful publication retains the current and previous complete SQL objects. Storage operations with an unknown completion outcome are not promoted. Export stored shows that the bytes were stored; it does not prove that a restore works.

Downloads accept only HTTPS Cloudflare R2 storage endpoints. Cloudflare's D1 export reference does not guarantee a download hostname; the first approved account check must confirm that boundary. Other hosts are rejected, not followed. On standalone installations and the protected Compose backup profile, the saved targets join the existing 04:00 UTC backup operation. Its fixed machine route accepts the installation's bootstrap bearer before store access; hosted and demo profiles reject that route. The host reads no provider token from the encrypted store and needs no asset checkout or Wrangler login. It verifies receipt identities, SQL bytes and hashes while streaming gzip into cloudflare-d1/<account>/<database>/ in the dated set. Each directory carries receipt.json with compressed hashes and a REST import pointer; backup.json records the whole selected set. Changed selection, unknown inventory or any failed target prevents publication, offsite handoff and retention. The previous complete set stays intact. Native and legacy exporters covering the same asset are rejected before either runs. Hosted exports remain manual. No provider is activated by committing this adapter. A real export needs the owner's approval for the exact account, database and verification scope: Cloudflare's running export temporarily blocks queries and must be polled until completion. The API download URL expires after one hour.

Provider-reported issues and recommendations

These are not interchangeable with traffic anomalies. A provider finding must be labeled with provider origin, retain the affected URL/site and provider evidence, and have first-seen/last-seen lifecycle fields so a disappeared issue auto-resolves without erasing history.

Provider	What is actually available	Practical NoticeOS approach
Bing Webmaster	GetCrawlIssues returns crawl-issue records for a site	Implemented in the daily archive. The latest snapshot can produce an executive warning with affected URLs and crawl evidence; promoting it into mutable Needs Attention lifecycle records still requires first-seen/last-seen deduplication
Google Search Console	No site-wide Recommendations/Issues feed appears in the API surface. URL Inspection returns indexing/fetch/robots/canonical/mobile/rich-result state for a requested URL	Inspect a bounded, evidence-selected priority URL set; never present it as a whole-site health scan
Google Analytics 4	The Data API reports metrics and the Admin API manages configuration/change history; neither documents a Recommendations/Insights feed	Continue deriving anomalies centrally from finalized reporting data; do not claim GA UI recommendations through the API

GSC URL Inspection should not be added until NoticeOS has an explicit priority-URL selector and quota budget. GA4 remains an evidence source for OS rules, not a provider-advice source.

Adding an integration (the checklist)

Purpose → consuming stage → auth + storage of creds (doc 06) → cost model + budget line → quota/latency/distortions entry (doc 02 table) → behaviour on failure (doc 06 table) → Tower setup-state tile → only then may hypothesis cards cite it.