A red team is only useful if it can stop you
Adding a challenge stage changes nothing if the challenge has no authority. Notes on giving an adversarial profile a real veto, and what broke when I did.
At first I had a challenge stage. It produced good objections. I read them, found them interesting, and proceeded exactly as I had intended to.
That is not a review process. That is a second opinion you have pre-decided to overrule, which is worse than none, because it leaves a paper trail that looks like diligence.
The change
One rule: the red team can stop a venture, and only written evidence restarts it. Not a better argument. Not my conviction. Evidence that addresses the specific objection raised.
What broke immediately
- Everything slowed down. Correctly. It turned out several things had been moving on my enthusiasm alone.
- I started pre-arguing. Writing cases aimed at surviving the red team rather than at being true. I caught it because the writing got defensive before it got specific.
- It over-fired. A profile whose job is objecting will object. A good share of the early stops were technically valid and commercially irrelevant.
The fix for over-firing
Not softening the veto. Narrowing what it applies to. An objection now has to name which surviving assumption it kills, and the assumptions are written down in advance at the verify stage.
“This could fail” is not an objection. Everything could fail. “This fails if assumption three is wrong, and here is why it is” is one.
That cut the noise substantially and made the remaining stops much harder to wave away.
The part I did not expect
The veto is mostly useful for what it prevents me from starting. Knowing an idea has to survive a written attack changes which ideas I bother to write up — and that filtering happens before any work is done, which is where filtering is cheap.