In our 2026-10-10 backtest of 100 needs, our "build vs.

In our 2026-10-10 backtest of 100 needs, our "build vs. skip" verdicts barely beat a coin flip — AUC 0.58, where 0.5 is random — so with a sample this small, I read it as an observation, not proof the verdict works.

The verdict said "build" on 8 needs and 2 of them later grew a product past 10,000 users within 12 months; it said "skip" on 92 and 13 succeeded. That's 25% vs. 14% against same-scale expectations of 0.75 and 7.76 — a 2.67× and a 1.68× lift (90% intervals 0–3.09 and 1.12–2.25) in our 2026-10-10 backtest.

The ranking did the work, not the verdict. Inside the top 20 by our evaluation standard, the needs the verdict still called "skip" lifted 1.74× (1.13–2.27), and the whole batch lifted 1.76× in our 2026-10-10 backtest — the verdict added almost nothing on top.

So we changed the question to "what's the chance a 10,000+ user newcomer appears here in 12 months?" In our 2026-10-10 backtest on the same 100 needs, that score hit AUC 0.88 (0.90 / 0.86). But a plain market-size ranking alone hit 0.92, and within the same size band our verdict scored only 0.70 (53 pairs) — the score mostly learned "bigger markets win".

Takeaway: rank ideas with a checkable standard, and use an AI's verdict only as a risk note — don't let it veto a market just because it looks crowded.

What standard do you use to rank what to build next?