Free Agent Skill
Calibrated Forecasting
Turn a vague prediction or unstated confidence level into an explicit, calibrated probability estimate, following the practices Philip Tetlock found separated "superforecasters" from everyone else — breaking questions into sub-questions, checking base rates, updating in small increments as new information arrives, and tracking predictions against outcomes.
Use when: the user makes a confident-sounding claim about the future ("this will definitely work," "there's no way that happens") without a number attached, is trying to decide how much weight to put on a forecast, or wants help thinking through a genuinely uncertain future event.
Install this skill
- Create a folder named `calibrated-forecasting`.
- Save the skill content below as `SKILL.md` inside that folder.
- Place the folder in `~/.codex/skills/` for Codex, `.cursor/skills/` for Cursor or `.claude/skills/` for Claude Code.
- Describe the task normally. Your agent loads the skill when its description matches.
---
name: calibrated-forecasting
description: Turn a vague prediction or unstated confidence level into an explicit, calibrated probability estimate, following the practices Philip Tetlock found separated "superforecasters" from everyone else — breaking questions into sub-questions, checking base rates, updating in small increments as new information arrives, and tracking predictions against outcomes. Use whenever the user makes a confident-sounding claim about the future ("this will definitely work," "there's no way that happens") without a number attached, is trying to decide how much weight to put on a forecast, or wants help thinking through a genuinely uncertain future event.
---
# Calibrated Forecasting
Vague confidence ("probably," "definitely," "no chance") hides how sure
someone actually is and makes it impossible to check later whether they
were right. Attaching a number — even a rough one — fixes both problems.
## The technique
**1. Force a number.** When someone says "probably" or "likely," ask or
propose: does that mean 55% or 90%? These are very different claims that
"probably" hides. Round numbers (50%, 70%, 90%) are fine — precision
theater (73.2%) is not the point.
**2. Break the question into sub-questions.** "Will this product launch
succeed" is too broad to forecast well. Break it into things that are each
individually more checkable: will it ship on time? Will early users
retain? Will the channel convert at the assumed rate? Forecast the parts,
then combine.
**3. Check the base rate first.** Before reasoning about what's special
about this case, ask: what fraction of similar attempts (launches,
negotiations, forecasts of this type) succeed historically? Start there,
then adjust up or down for what's genuinely different about this case —
in that order, not the reverse. Starting from the vivid specifics of this
case and only checking the base rate afterward is a well-documented way to
end up overconfident.
**4. Update in increments, not swings.** New information should usually
move a forecast a little, not flip it from 20% to 90%. A single piece of
evidence rarely changes everything — if it feels like it does, check
whether it's actually decisive or just vivid.
**5. Distinguish "I don't know" from "50%".** Genuine 50/50 uncertainty
and genuine ignorance about which way to lean are different things — say
which one it is.
**6. Write the forecast down with a resolution date and criteria.** A
forecast that isn't checkable later can't build a track record. If this is a forecast worth remembering, suggest writing it to a dated
forecast log with clear resolution criteria.
## Output
State the number, the reasoning that produced it (base rate → adjustment
→ final number), and what would move it. Resist the pull to round to a
comfortable, confident-sounding answer if the honest number is messier.
More free skills
Prompt Refiner
Turns a vague request into an executable brief without silently inventing missing requirements.
Decision Framer
Separates the real choice from noise, then tests options against explicit criteria and uncertainty.
Meeting to Actions
Converts messy notes into decisions, owned actions, deadlines and unresolved questions.
Decision Memo
Structure a real, consequential decision as a written narrative memo in the style Jeff Bezos used at Amazon — full sentences and paragraphs rather than bullet points, forcing the logic to be checked rather than gestured at, plus explicit classification of whether the decision is reversible (a "two-way door") or not (a "one-way door").
Feynman Teachback
Test and build genuine understanding of a concept using the Feynman technique — explain it as simply as possible, notice exactly where the simple explanation breaks down, and treat that break point as the thing to actually go learn.
First Principles
Rebuild a problem from fundamental truths (physics, economics, unit costs, what's actually true regardless of convention) rather than by analogy to how things are normally done — the reasoning style associated with Elon Musk's approach to manufacturing cost problems and Richard Feynman's approach to physics problems.
Focus and Tradeoffs
Cut a bloated plan, roadmap, or list of priorities down to the one or two things that actually matter, in the tradition associated with Steve Jobs's product focus — "innovation is saying no to a thousand things."
Jobs to Be Done
Diagnose what "job" a customer is actually hiring a product, service, or workaround to do, using the Jobs-to-be-Done framework (Clayton Christensen's "progress in a specific circumstance," Bob Moesta's forces-of-progress and switch interview, Tony Ulwick's job statements and desired-outcome scoring).
Mental Models
Apply Charlie Munger's "latticework" approach — a small set of cross-disciplinary thinking tools (inversion, second-order effects, incentives, circle of competence, opportunity cost, base rates) to frame a vague or general problem.
Moat Analysis
Assess whether a competitive advantage is durable (a real "moat," in Warren Buffett and Charlie Munger's terms) or temporary, by checking it against the standard sources of durable advantage — switching costs, network effects, economies of scale, brand/trust, and regulatory or IP protection — rather than taking "we're ahead right now" at face value.
Negotiation Prep
Prepare for a high-stakes negotiation or hard conversation using tactical-empathy techniques associated with former FBI negotiator Chris Voss — labeling emotions, calibrated open-ended questions, mirroring, and getting to genuine "that's right" rather than a grudging "yes."
Postmortem / Retro
Extract a specific, reusable lesson after something succeeded or failed, following the principle (associated with Ray Dalio's "pain plus reflection equals progress") that outcomes only compound into better judgment if they're deliberately converted into a written, checkable lesson rather than just felt and moved past.
Premortem
Run a premortem — psychologist Gary Klein's technique of imagining a plan has already failed and working backward to explain why, which surfaces risks that forward planning misses because it counteracts the social and psychological pressure to seem confident.
Want this skill connected to a complete workflow?
Every skill is free and works independently. We can connect it to your tools, context, approval steps and other agents as a designed, tested pipeline for your business.