door ingest: anchor card matching to the HeatSync hex-uid model
The inbound-ingest matcher read a stored card_uid as both decimal and hex, which mis-attributed all-digit uids (a stored "16" is hex 0x16 = 22, but it wrongly matched a decimal poll of card 16) and was conceptually wrong. HeatSync stores the uid as hex (the grant encoder requires it) and the ?z log / webhooks report the decimal card number, so cardMatchesEvent now matches a reported decimal when hexInt(storedUid) equals it, and matches a webhook card_uid by exact string. This removes the false positive while keeping every legit match. Attribution is audit-only (never an access decision) and inert until a space enables ingest; no migration. Also records the pass-63/64 door-epic deploy state and the pass-65 audit in the handoff log.
This commit is contained in:
@@ -939,7 +939,7 @@ Phase 3 live actions (all `door.operate`, rate-limited, every attempt writes one
|
||||
|
||||
Member self-entry (no permission code; any active member): `selfEntry({connectionId})` triggers a momentary OPEN only, allowed only when the connection is `is_enabled` AND `allow_member_self_entry` AND the caller has at least one active `member_card` on file (the locked eligibility rule: a `door_card_slots` row is NOT required). Membership and cards are resolved server-side, so a member can only ever open a door for themselves; never unlock/lock/grant/revoke, never anonymous. Strict per-member rate limit (5/min), one redacted `door_access_log` row `action='self_entry'` (`target_member_id` = self), through the same hardened executor. `listSelfEntryDoors()` returns the enabled self-entry connections for the caller's space, but only when the caller has an active card (otherwise empty), so the surface is hidden for ineligible members. `listMyDoorActivity()` returns the caller's own `door_access_log` rows (actor or target; service client after `requireMember`; `detail` is already secret-redacted at write time). Surfaced as a "Door access" panel on the dashboard and on the member `/doors` page (self-entry + masked own cards via `getMyCards` + recent personal activity).
|
||||
|
||||
Phase 4 inbound access-log ingest (migration 053): real entry/denied events are pulled INTO `door_access_log` and matched to a member, via two transports sharing one ingest core (`lib/door/ingest.ts`, service-client writer; the audit log stays no-client-write). Config is folded into `updateDoorConnection`/`createDoorConnection`, which gained `inbound_enabled` (gates both transports, off by default) and `inbound_secret_ref` (a distinct vault secret from the outbound `secret_ref`; the create/update path verifies every referenced secret belongs to the caller's space). Both transports resolve a presented card to a member by numeric hex/decimal equality (`cardUidsEquivalent`) and dedupe-insert: `door_access_log.dedupe_key` + the partial-unique `(connection_id, dedupe_key)` index make a re-poll / webhook retry a no-op; action rows leave it NULL.
|
||||
Phase 4 inbound access-log ingest (migration 053): real entry/denied events are pulled INTO `door_access_log` and matched to a member, via two transports sharing one ingest core (`lib/door/ingest.ts`, service-client writer; the audit log stays no-client-write). Config is folded into `updateDoorConnection`/`createDoorConnection`, which gained `inbound_enabled` (gates both transports, off by default) and `inbound_secret_ref` (a distinct vault secret from the outbound `secret_ref`; the create/update path verifies every referenced secret belongs to the caller's space). Both transports resolve a presented card to a member via the HeatSync hex-uid model (`cardMatchesEvent`: a stored hex `card_uid` matches a reported DECIMAL number when `hexInt(uid)` equals it; a webhook `card_uid` matches by exact string) and dedupe-insert: `door_access_log.dedupe_key` + the partial-unique `(connection_id, dedupe_key)` index make a re-poll / webhook retry a no-op; action rows leave it NULL.
|
||||
- `POST /api/cron/door-ingest` (nodejs, force-dynamic; constant-time `CRON_SECRET` bearer, 503 if unset / 401 on mismatch; `proxy.ts`-whitelisted). Polls each `is_enabled` + `inbound_enabled` + `adapter='native_heatsync'` connection's `?z` log through the hardened executor (`fullBody`, 32KB cap), parses it with the pure firmware-characterized `lib/door-log-logic.ts` (`parseHeatSyncLog`: `<pre>` ring-buffer dump, G+g/D+d card-number reconstruction via the 32767 divisor, best-effort H:M:E, raw R/r reads intentionally skipped), and ingests. Bounded to 50 connections/run. Returns `{ ok, polled, inserted, resolved, failed }`. Best-effort by nature (the HeatSync ring buffer has no per-entry sequence id); idempotent across overlapping/missed minutes.
|
||||
- `POST /api/door/inbound/[connection]` (nodejs, force-dynamic; session-exempt, `proxy.ts`-whitelists the `/api/door/inbound` prefix). Authenticated by the connection's per-connection bearer secret (`inbound_secret_ref` from the vault, constant-time compared; the same generic 401 for not-found / inbound-disabled / wrong-secret so it is not an existence oracle). Body-size guarded (413 over 64KB), per-connection rate-limited (120/min), Zod-validated (`doorWebhookPayloadSchema`: `{ events: [{ id, card_uid?|card_number?, result?, occurred_at? }] }`, ≤100 events; `id` is the dedupe token). `normalizeWebhookEvents` → ingest core. A supplied `occurred_at` is clamped to not-in-the-future (60s skew) so a relay cannot reorder the operator's log. Returns `{ received, inserted, resolved, skipped }`. This is the reliable inbound transport for any (incl. generic/non-HeatSync) controller.
|
||||
|
||||
|
||||
@@ -630,8 +630,10 @@ H:M:E). Webhook: `POST /api/door/inbound/[connection]` (also
|
||||
controller/relay, authenticated by a per-connection bearer secret
|
||||
(`inbound_secret_ref`, a distinct vault secret from the outbound password,
|
||||
constant-time compared); body-size guarded, rate-limited, Zod-validated
|
||||
(≤100 events). Both resolve a presented card to a member via numeric
|
||||
hex/decimal equality (`cardUidsEquivalent`) and dedupe-insert through the
|
||||
(≤100 events). Both resolve a presented card to a member via the HeatSync
|
||||
hex-uid model (`cardMatchesEvent`: a stored hex `card_uid` matches when
|
||||
`hexInt(uid)` equals the reported decimal number, or a webhook uid matches
|
||||
exactly) and dedupe-insert through the
|
||||
service client: `door_access_log.dedupe_key` + the partial-unique
|
||||
`(connection_id, dedupe_key)` index make a re-poll or webhook retry a
|
||||
no-op, while NULL-keyed action rows stay unconstrained. The poll is
|
||||
|
||||
+22
-5
@@ -28,7 +28,24 @@ Append-only. Newest entries on top. Keep each entry to one screen.
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-20 (pass 64): Door epic Phase 5 (universal API-call UI builder) (LOCAL, NOT deployed)
|
||||
## 2026-05-20 (pass 65): Door epic P4+P5 DEPLOYED + ultrathink audit
|
||||
|
||||
Branch `main`. Shipped the batched door-epic completion and ran a fresh diversified audit.
|
||||
|
||||
**Deploy (DEPLOYED).** Passes 63 (P4 inbound ingest) + 64 (P5 API-call builder) shipped together in `7ec90b0` (run 26204184395, success). Deploy log confirms `applying 053_door_inbound_ingest.sql` then `applying 054_api_buttons.sql`, no errors. Smoke: `/` 200, `/login` 200, `/dashboard` `/doors` `/door/manage` `/door/buttons` 307->login (the new builder route exists + is gated, not 404). New endpoints fail-closed: `/api/cron/door-ingest` 503 (CRON_SECRET still unprovisioned in prod -> poll inert), `/api/door/inbound/<malformed>` 404, `/api/door/inbound/<uuid>` no-bearer 401. Could not run the prod anon-key auth-embed (PGRST201) check from here (local `.env.local` holds the local-stack key, not the prod JWT); the regression class is covered by `integration/auth-embed.test.ts` (green vs the migrated 053+054 schema) and the schema reasoning (door_connections' 2nd secrets FK is a surrogate-PK table, not a junction; nothing embeds secrets; the auth embed is untouched). `/dashboard` 307 (no `/signup` bounce) is consistent with a healthy embed.
|
||||
|
||||
**Ultrathink audit (fresh, diversified; no auto-defend).** Surfaces: public/API, authz/RLS, XSS/rendering, schema/PGRST, the shared egress refactor, `logActivity`.
|
||||
- **Re-verified clean.** The egress refactor preserves `callDoor` exactly (the SSRF pin is enforced on the FINAL assembled URL for `callApi` too: base + url_template + query-secret -> `validateDoorTarget` -> resolve-once -> connect-by-IP; host header forced to the pin; metadata/link-local block is absolute even for an admin-configured button; url_template cannot inject a host past the pin). AuthZ/tenant isolation: button loads are space-pinned, invoke is per-row-permission-gated, the denial path is now rate-limited. XSS: all new rendering is escaped JSX. PGRST: no new junction (all new tables have surrogate PKs; api_buttons/api_call_log are never embedded). `activity_log` is free-text so `logActivity('api_button', ...)` inserts cleanly. The `apicall.invoke` seed `CREATE OR REPLACE` is idempotent + backfilled.
|
||||
- **One finding, FIXED this turn (was LOW; audit-only; inert).** The original `cardUidsEquivalent` matched a card uid by exact string OR numeric equality across BOTH decimal and hex readings of the STORED uid, which both mis-attributed all-digit uids (stored `"16"` = hex 0x16 = 22 wrongly matched a decimal poll of card 16) and was conceptually wrong. Replaced with source-anchored `cardMatchesEvent(storedUid, ev)`: HeatSync stores the uid as hex (the grant encoder requires it) and `?z`/webhook report the DECIMAL card number, so a reported decimal matches when `hexInt(storedUid)` equals it; a webhook `card_uid` also matches by exact (case-insensitive) string. The stored uid is never read as decimal, removing the false positive while keeping every legit match (hex `9c40` <-> poll `40000`; all-digit hex `10` <-> poll `16`). +tests; suite 626. Note for whoever enables ingest: this assumes stored uids are hex (true for the HeatSync grant path); confirm against a real controller, and if a space stores DECIMAL uids the anchoring would need revisiting.
|
||||
- **Accepted trade-offs (re-confirmed, documented).** Webhook decrypt-flood needs an unguessable connection uuid (post-auth rate limit kept to avoid a targeted relay lockout); the webhook content-length guard is best-effort behind the platform body limit; the cron automates an admin-configured outbound call within the existing door-connection trust model; audit inserts are best-effort (a call result still returns on a logging blip).
|
||||
|
||||
**Backlog item 5 TRIAGED + CLOSED: `user_has_permission` EXECUTE-to-PUBLIC.** Investigated the grant flagged pass 56. Findings: (1) `user_has_permission` (+ `user_has_role_in_space`) grant EXECUTE to `anon` + `authenticated` + `service_role` (explicit, not loose PUBLIC). (2) `anon` holds the DEFAULT Supabase broad grants (SELECT/INSERT/UPDATE/DELETE) on all 51 public tables; RLS is the SOLE gate, and ~40 policies call `user_has_permission` in their USING/CHECK. So `anon` genuinely EVALUATES those policies, which means revoking EXECUTE from `anon` would make anon queries **error** ("permission denied for function") rather than deny -- it is STRUCTURALLY REQUIRED, not removable. (3) The residual exposure is a boolean oracle (`rpc user_has_permission(uid,sid,perm)` -> does uid hold perm in sid), but it needs a known `auth.uid` (a victim's UUID), which `anon` cannot obtain (auth ids are not anon-readable; single-space-per-user makes cross-space probes mostly `false`). (4) The genuinely dangerous SET-returning enumeration variant (`members_with_permission`) is already locked (`REVOKE FROM PUBLIC` / `GRANT service_role`, migration 047). **Conclusion: no change; the grant is required by the RLS model and the practical leak is a low-value boolean gated by uid secrecy.** A harder lockdown (moving the function to a PostgREST-unexposed schema so it stays RLS-callable but is not a public RPC) is a large, risky refactor touching every policy and is NOT justified by the residual risk. Item 5 closed.
|
||||
|
||||
**State.** Suite 626 unit / 50 integration / build + lint green. Migrations through 054. Door epic feature-complete (P1-P5). Backlog item 5 closed (no change). Ingest attribution matcher hardened (`cardMatchesEvent`).
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-20 (pass 64): Door epic Phase 5 (universal API-call UI builder) (DEPLOYED in 7ec90b0; see pass 65)
|
||||
|
||||
Branch `main`. Completes the door epic. Design-first via AskUserQuestion; the user chose **full verbs** (GET/POST/PUT/PATCH/DELETE) + **per-button required permission**. Suite 624 unit (was 615; +9 buildApiRequest tests) / 50 integration (was 44; +6 api-buttons) / build green / lint clean.
|
||||
|
||||
@@ -39,13 +56,13 @@ Branch `main`. Completes the door epic. Design-first via AskUserQuestion; the us
|
||||
- **UI**: builder `/door/buttons` (grouped list, create form, door-template preset, secret + required_permission pickers, enable/delete, the call log) + sidebar "API buttons" link (door.manage). Member invoke: an "Actions" section on `/doors` (grouped buttons, confirm dialog).
|
||||
- **Audit**: SSRF egress reused (host pin + metadata block absolute even for admin config; member never supplies url/headers/body, only a buttonId); tenant-isolated; secret space-scoped + never returned + redacted; all rendering escaped JSX; single secrets FK (PGRST safe, auth-embed green). **Fix:** moved the invoke rate-limit before any DB work so a member can't flood denial-audit rows.
|
||||
|
||||
**Deploy state.** NOT deployed; **batched with the pass-63 (P4) work** awaiting one deploy authorization. Migrations 053 + 054 applied to the LOCAL stack only. No new env var or cron for P5 (all in-app). Inert until an admin creates a button.
|
||||
**Deploy state.** DEPLOYED in `7ec90b0` (run 26204184395; see pass 65). Migrations 053 + 054 live on prod. No new env var or cron for P5 (all in-app). Inert until an admin creates a button.
|
||||
|
||||
**Open / next.** Door epic is feature-complete (P1-P5). v1 limits (documented, not bugs): the builder supports create/enable/delete (full field edit = delete+recreate); buttons use static url/headers/body (no per-press placeholder substitution). Browser click-through of `/door/buttons` + the `/doors` Actions section still pending (auth-gated). Back to the main backlog after this: owner-gated spine validation, observability, etc.
|
||||
|
||||
---
|
||||
|
||||
## 2026-05-20 (pass 63): Door epic Phase 4 (inbound access-log ingest) (LOCAL, NOT deployed)
|
||||
## 2026-05-20 (pass 63): Door epic Phase 4 (inbound access-log ingest) (DEPLOYED in 7ec90b0; see pass 65)
|
||||
|
||||
Branch `main`. Took backlog item 6. Design-first via AskUserQuestion; the user chose BOTH transports. Built poll + webhook sharing one ingest core. Suite 612 unit (was 594; +18 parser tests) / 44 integration (was 40; +4 door-ingest) / build green / lint clean.
|
||||
|
||||
@@ -53,13 +70,13 @@ Branch `main`. Took backlog item 6. Design-first via AskUserQuestion; the user c
|
||||
|
||||
**Shipped (local):**
|
||||
- **Migration 053** (additive only): `door_access_log.dedupe_key` + partial-unique `(connection_id, dedupe_key)` (re-poll / webhook-retry no-op; action rows stay NULL/unconstrained); `door_connections.inbound_enabled` (opt-in, off) + `inbound_secret_ref -> secrets` (inbound bearer secret, distinct from the outbound password). schema.sql + types + DATABASE_SCHEMA + DB_SCHEMA_MAP same-change. PGRST: 2 secrets FKs but surrogate PK = not a junction (safe re 052); auth-embed test still green.
|
||||
- **Pure parser** `lib/door-log-logic.ts` (`parseHeatSyncLog`, `normalizeWebhookEvents`, `cardUidsEquivalent` hex/decimal match) + 18 unit tests.
|
||||
- **Pure parser** `lib/door-log-logic.ts` (`parseHeatSyncLog`, `normalizeWebhookEvents`, `cardMatchesEvent` source-anchored hex-uid match -- see pass 65) + unit tests.
|
||||
- **Ingest core** `lib/door/ingest.ts` (resolve card->member, dedupe-insert via service client). Extracted the shared `resolveDoorSecret` to `lib/door/secret.ts` (door.ts now imports it). Executor gained an additive `fullBody` option (32KB cap; existing callers unchanged at 4096/500).
|
||||
- **Poll** `POST /api/cron/door-ingest` (CRON_SECRET, proxy-whitelisted, native_heatsync + inbound_enabled only, ≤50 conns/run). **Webhook** `POST /api/door/inbound/[connection]` (proxy-whitelisted prefix, per-connection bearer constant-time, generic 401 non-oracle, 64KB body guard, 120/min, Zod ≤100 events). Both feed the ingest core.
|
||||
- **Admin UI**: per-connection "Inbound" panel on `/door/manage` (secret picker, on/off toggle that requires a secret first, copyable webhook URL + payload hint, "inbound on" badge).
|
||||
- **Audit hardening** (3 passes): body-size guard before JSON parse; `occurred_at` clamped not-in-future so a relay can't reorder the log. **Fresh-pass real bug fixed:** the `?z` body is untrusted plaintext HTTP, and `Number('9'*400)===Infinity` then `BigInt(Infinity)` THROWS, which would 500 the whole poll and starve other connections; capped the parser value at 15 digits (firmware is int32) + bounded the G/g pairing window + wrapped each connection in the cron loop in try/catch. Verified: executor change behavior-identical for existing callers; inbound secret space-scoped on write + read; tenant isolation from the looked-up connection; ingested `detail` renders as escaped JSX (no XSS); audit log stays immutable from the client (integration-tested). Accepted trade-offs (documented): decrypt-flood needs an unguessable uuid; `resolved` count is mild info to the already-trusted relay; the content-length guard is best-effort behind the platform body limit.
|
||||
|
||||
**Deploy state.** NOT deployed; awaiting the per-request go (ASK-before-deploy). Migration 053 is applied to the LOCAL stack only. No new env var (reuses CRON_SECRET); the door-ingest crontab line is documented in DEPLOYMENT but optional (only spaces that enable inbound need it). Inert until an admin turns on `inbound_enabled` for a connection.
|
||||
**Deploy state.** DEPLOYED in `7ec90b0` (run 26204184395; see pass 65). Migration 053 live on prod. No new env var (reuses CRON_SECRET); the door-ingest crontab line is documented in DEPLOYMENT but optional (only spaces that enable inbound need it). Inert until an admin turns on `inbound_enabled` for a connection AND the owner provisions CRON_SECRET + the crontab.
|
||||
|
||||
**Open / next.** Door epic **Phase 5** (universal API-call UI builder, `api_buttons`) is the remaining door work; it needs its own design pass (HTTP-method scope + per-button vs fixed invoke permission forks) and extends the executor to method+headers+body. Browser click-through of the `/door/manage` inbound panel still pending (auth-gated; can't curl). Minor deferred: a far-future-date relay can't reorder but a near-now spoof is accepted (trusted relay); `resolved` count is mild info to the trusted relay.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user