Expertise Triage Framework
A fast, repeatable way to assess credibility in real time
Core Methods
1. Canary Test (Early Warning – Negative Update)
- Definition: Spot a clear error, flawed logic chain, or internal contradiction — especially in an area where you can reason from first principles.
- Why it matters: If they miss something obvious here, there may be hidden flaws elsewhere.
- Action: Downgrade trust until proven otherwise.
2. Gold Nugget Test (Positive Update)
- Definition: Identify an idea that’s both novel and holds up under scrutiny.
- Why it matters: Suggests genuine, independent thinking and domain mastery.
- Action: Upgrade trust and watch for more nuggets.
3. Track Record Analysis (Gold Standard)
- Definition: Compare past predictions to actual outcomes, focusing on falsifiable claims.
- Why it matters: Objectively measures calibration and accuracy.
- Action: Prioritise voices with proven, transparent accuracy over time.
Modifier Methods
4. Incentive Mapping (Bias Context)
- Definition: Ask, “Who benefits if this claim is believed?”
- Use carefully: Reveals possible bias but doesn’t prove falsehood.
5. Source Triangulation (Corroboration Check)
- Definition: See if multiple independent credible sources converge on the same claim.
- Watch out: Avoid false consensus caused by echo chambers or shared sourcing.
How to Apply in Practice
- Listen actively for Canaries (errors) and Gold Nuggets (novel, sound insights).
- If time allows, check historical claims for track record scoring.
- Map incentives only to interpret motive, not truth value.
- Use triangulation to fill gaps when direct verification is impossible.
- Adjust weight you give the source:
- Many Canaries + no Gold Nuggets → Distrust.
- Few Canaries + frequent Gold Nuggets + strong track record → High trust.
Rule of Thumb:
One strong Canary can outweigh many average claims.
One strong Gold Nugget can earn a source the benefit of the doubt — until a Canary appears.
Track record, when available, trumps all.