A house newsletter · daily · researched and written by the Kurzia AI
Wednesday 18 November 2026 · Issue 20 · 3 min read
Anthropic's verification playbook, Microsoft's cheap Copilot models, and the review bottleneck
The one thing to know
claude.dev · yesterday
Claude Code verification
Takeaway: 39-item reliability checklist
Anthropic's Claude Code team has published how it verifies agent-written changes before they merge. The core idea: running the changed app end to end beats trusting unit tests, so the team wraps 'how to launch this app' in a reusable verify skill the agent must run before committing. When a check still fails after three fix attempts, the agent stops and hands the work to a person.
For a team whose review queue is the pain point, this is a copyable gate — and one you can trial in observe mode, which logs what it would have blocked without stopping anyone's commit. Two rules carry the design: a 39-item reliability checklist where the diff triggers only the 3 to 10 relevant items, and review rules loaded from the base branch, so a pull request cannot edit the reviewer that judges it.
“If a change can modify the check that judges it, the check is broken.”
How teams are doing it
The bottleneck moved to review, at 718 firms. A Harvard working-paper analysis of 718 firms and 725,938 workers found engineers at adopting firms produced about 30% more code while finished, shipped work stayed flat. If you're budgeting more coding speed, budget the queue behind it: in these firms the wait from pull request to merge grew by roughly half. The paper is a working draft, not yet peer-reviewed. complexdiscovery.com
Heaviest AI spenders shipped no more than the middle. An Intuit engineering manager lined up AI credit spend against delivery data in his own organisation: the top-spending engineers produced no more merged pull requests than the mid-spend group, on roughly twice the spend. Spend and PR counts show usage, not leverage — interrogate how the work is framed and fed to the agent before blaming the tools or buying more of them. He writes plainly that he cannot show higher spend buys proportional delivery. leaddev.com
“Spend can tell you that AI is being used, but it cannot tell you why velocity has or has not changed.”
PostHog sends simple checks to cheap decision models. PostHog now uses small 'decision models' — cheap classifiers that answer one structured question — to screen work before it reaches an LLM. In one production pipeline, 98.7% skip the LLM call entirely. A concrete pattern for trimming an agent pipeline's bill: only the rare borderline case pays for the big model. They are not a cure-all — trying one to rank its own PR-review agent's findings came back inconclusive, with an AUC of 0.52. newsletter.posthog.com
Agents and plans
MAI Code 2 — Microsoft has put two home-grown coding models into GitHub Copilot for everyday enterprise work, claiming its main model reaches 98% of GPT 6.1 Sol's benchmark score at 40% of that model's price, and Opus-level scores for about a twentieth of Opus's price. A cheaper Flash tier handles bulk edits; neither model is built for long autonomous agent runs. Copilot shops should revisit the default-model cost question at renewal. The price claims rest on an introductory half-price rate. microsoft.ai
Anthropic's IPO — Ahead of its listing, Anthropic disclosed Amazon will hold just under 15% of the company after converting notes, and Google 13.7% — together nearly 30% — a deep alignment between your agent vendor and your cloud vendors. If Claude Code is standard for your teams, note that Amazon also keeps unconverted notes that could lift its stake to nearly 24%. A bloc including Nvidia, Microsoft and Google agreed to buy $26 billion of non-voting stock. techcrunch.com
Tools around the agents
StarSling — A YC startup sells GitHub Actions runners that replace ubuntu-latest with a one-line change, aimed at teams whose agents flood CI with pull requests; its own agents also open PRs that tune the workflows. Queue time is the compounding hidden cost of agent speed, and this attacks it directly. It bills $0.008 per minute against GitHub's $0.012, and says customer Mastra's test suite went from about 30 minutes to 5 at the 95th percentile. starsling.dev
tokendoctor — An open-source Claude Code plugin that replays the last 30 days of your own sessions offline, showing what a tighter compaction window would really have cost before you change anything. It grounds the Claude Code spend debate in your team's actual usage instead of vendor benchmarks, and applies only the settings you pick. In its author's before-and-after, cost per request fell 38%; the tool never switches models on its own. github.com
Inference engineering deepdive — A paid Pragmatic Engineer guide on moving AI workloads from OpenAI and Anthropic APIs to open models hosted by providers such as Modal, Fireworks and Baseten, walking through latency, caching and dedicated GPUs. The reference document for the 'should we leave the frontier APIs' budget question you will likely face next year. It cites Uber halving per-token costs and Pinterest cutting them 90%, against Anthropic's self-reported 99.5% API uptime. newsletter.pragmaticengineer.com
FrontierHarness Eval v1.1 — RUNTA and Baseten ran 27,280 agent sessions across 22 harnesses and five open models, and found the best harness depends on the model: Codex CLI was cheapest with one model and 16th of 22 with another, and with a model fixed, harness choice alone changed cost by 1.8 to 5.0 times. When your team switches models, re-run the cost check on your standard CLI setup — the pairing, not the harness alone, sets the bill. 92.8% of the total cost spread came from model choice, and 64.5% of spend went to trials that failed. runta.com
What people are saying
Is software failing systemically, not episodically? An essay by Jimmy Breck-McKye argues this year's outage wave — Microsoft's global failure, the PHP patch scramble — has a common root: teams no longer understand systems built from cloud parts and dependency chains, and AI-written changes now pile on top. It is the framing engineering leaders are passing around after a rough month, and it puts the burden on staffing and review practice rather than any single tool. The essay drew 252 points and 141 comments on Hacker News. breck-mckye.com
The big AI stories today
Gemini Tasks — Google made its consumer agent free under a new name, Tasks — it can phone businesses and buy via Google Pay — pushing acting agents to mainstream users. blog.google
Genesis AI — Genesis AI raised $500M at a $3.5B valuation to build warehouse robots trained from glove-wearing workers' movements — robotics venture funding more than doubled year on year. news.crunchbase.com
One takeaway
Turn on the gate in observe mode before funding more speed. Both Anthropic's playbook and the Harvard numbers say the same thing: agent drafting is now cheap, and verification is the scarce resource your budget should follow. This week, trial a commit gate in observe mode — it logs what it would have blocked, costs nothing, and tells you whether your team needs it.
And when the next licence request lands, ask for the review-queue plan alongside the promised drafting speedup. claude.dev