Deployable task coverage × Alignment/assurance

Because these systems are narrow enough to bound and inspectable enough to insure, they get sold less as intelligence than as certified instruments — closer to a medical device than software — and insurers, not capability, end up deciding which tools become normal. The valuable expertise inverts too: the money is in knowing exactly where the tool stops working, a skill so tacit and unscalable that the productivity dividend everyone expected from 'safe AI' never fully shows up. And the audit trail that makes it trustworthy is also forensic exhaust — everything inspectable is everything subpoenable.

This is the corner everyone treats as the finish line, which is exactly why nobody interrogates it: a system broadly capable and fully correctable still doesn't answer who holds the correction rights, and a perfectly corrigible system obeys whoever can patch it — including a state pointing the rollback switch at its own citizens. The dependability half of the bargain cashes itself in quietly too: organizations stop maintaining the human skill to act on a failed audit, so the system stays inspectable while the people meant to fix what it reveals have forgotten how, and using the checkable system becomes a legal obligation the moment it exists.

The redeeming twist is structural, not moral: because these systems fail loudly and often, nobody gets the chance to build heavy dependence on them — you can only over-trust a tool that usually works, and this one visibly doesn't. What it corrodes instead is the ability to tell expertise from ritual: users accumulate lucky prompts and taboo inputs that look like professional judgment but are really pattern-memory around an unstable tool, and a market in fake assurance — dashboards, certifications, explainability theatre — fills the gap the demand for confidence leaves behind.

The danger isn't that this system is unreliable — it's that it's dependable enough to earn full delegation and still brittle under exactly the conditions — a crisis, an attack, a genuinely novel situation — where everyone needed it to hold. Because dependable systems converge on the same handful of foundation models, institutions that look independent often share the same invisible blind spot, so failure isn't scattered, it's simultaneous — and the longer the system is right, the less anyone retains the habit of asking why.