Blind Audit: Vanta Desktop

2026-07-18 - Full 16-framework sweep - audit only, no automatic product or roadmap mutation

Verdict: the app is substantially functional and does not need another wholesale shell rewrite. It is not release-clean because exact text is corrupted, the live proof hides that defect behind a timeout, drafts cross task boundaries, and several release claims remain fixture- or environment-limited.

Priority findings

Critical

Exact chat text is mutated

VANTA_DESKTOP_AUDIT_... renders as VANTADESKTOPAUDIT_.... The shared underscore-emphasis regex affects user and assistant bubbles, threatening file names, environment variables, IDs, hashes, and exact-output work.

High

The live proof fails opaquely

The official proof waits 150 seconds for already-corrupted rendered text and does not dump raw response, rendered messages, provider, root, approval, or errors. A separate diagnostic proved the provider and isolated root work.

High

Draft ownership is unsafe

One volatile React draft follows session changes and disappears on reload. Drafts need project + session scope, persistence, and explicit keep/discard/move behavior.

High

The current local candidate is unnotarized

The app is validly signed but Gatekeeper rejects it as Unnotarized Developer ID. The published notarized 0.9.2 CI artifact is separate from the changed local candidate.

Medium-high

Provider model and local runtime model conflict in copy

The same screen can show gpt-5.5 and No model. Both can be technically true, but the interface does not explain the two layers.

Medium-high

Accessibility is structured, not fully proven

Keyboard, roles, focus, contrast, zoom, and reduced motion have coverage. Automated axe scans and a physical VoiceOver flow do not.

Medium

No pixel or performance release budget

Geometry tests do not catch every visual regression. The app is about 548 MB on disk, and no cold-start, memory, CPU, or package-size threshold is enforced.

Medium

External coverage is narrow

Telegram passed live connect/poll/disconnect, but no complete inbound-message to agent-response round trip was executed. Twenty-one other channels correctly report not configured.

Medium

Declined crypto code ships in the default app

The operator explicitly parked x402, yet @x402/core and @x402/evm remain direct dependencies and are present in the packaged app.

Medium

The roadmap has no intake for the new defects

The roadmap reports 1,247 shipped and 11 parked cards. None currently represents message fidelity, draft ownership, or live-proof diagnostics.

What passed

All 16 lenses

FrameworkResult
Bias blind spotRoadmap completion creates an incentive to confuse shipped cards with current defect absence.
CalibrationExecuted, fixture/code-path, and unverified claims were separated.
Chesterton's fencePreserve the documented purpose of runtime, inspector, and dormant boundaries before removing them.
Consider the oppositeThe core app is coherent; targeted trust repairs beat another shell rewrite.
Curse of knowledgeTools 141, MCP 0, gateway, kernel, provider model, and runtime model assume internal vocabulary.
Dunning-KrugerNo unexecuted claim is made for screen readers, all providers, or every connector.
FalsificationThe underscore marker disproved exact-render fidelity.
InversionTrust fails through text mutation, wrong-task drafts, ambiguous model state, and blocked releases.
Johari windowFresh-context human and physical assistive-technology feedback remain absent.
Ladder of inferenceOld broken-shell screenshots were not treated as current behavior; a fresh launch replaced that assumption with direct evidence.
Outside viewPackaging, auth drift, external setup, and state recovery are typical late failures and remain priority proof areas.
PremortemLikely launch incidents are corrupted identifiers, wrong-task execution, Gatekeeper rejection, and connector delivery failure.
Red teamElectron isolation is strong; package/dependency surface and proof gaps remain.
SteelmanThe requested Codex/Hermes foundations already exist and should be preserved.
Survivorship biasFixtures and configured Telegram are visible; unconfigured accounts, offline drafts, AT, and changed release candidates are not.
Unknown unknownsDraft scope, dual-model terms, package composition, release binding, budgets, and provider drift surfaced.

Sequence

  1. Fix message fidelity and split raw/rendered live-proof assertions.
  2. Persist drafts by project and session.
  3. Clarify agent model versus local runtime; run one cold-user task.
  4. Add axe, VoiceOver, and pixel-regression gates.
  5. Bind notarization and real connector packets to the exact release commit.
  6. Decide whether dormant x402 belongs in the default desktop package.

Artifacts

User-flow report - Detailed Markdown blind audit