That realization, the slow recognition that I’d been wrong in the same way twice, was the catalyst for a habit I now consider the highest-leverage practice in my toolkit. For years, I operated the way most CTOs operate: trusting a blend of experience, intuition, and ad-hoc deliberation to make calls, with zero ability to distinguish the domains where my judgment was sharp from the ones where it was consistently miscalibrated. The decision journal changed that, with almost no overhead once the habit took root.
Why Intuition Alone Fails at Scale
Experienced leaders develop strong intuitions, and those intuitions are valuable. The problem is that intuition is a black box with no feedback loop. You make a call, time passes, the outcome arrives wrapped in so many confounding variables that you can’t cleanly attribute it to the original decision, and your brain does what brains do: it rewrites the narrative. You remember being less confident than you were, or more confident, depending on the outcome. Psychologists call this hindsight bias, and it’s especially corrosive for people in senior roles because the stakes are higher and the feedback cycles are longer.
A decision journal is the simplest corrective. It creates a written record that your future self cannot revise: a snapshot of what you believed and why at the moment you committed. The mechanism is trivially simple, and that simplicity is the point.
What to Record
The format matters less than consistency, but after several years of iteration, here’s the structure I’ve settled on for each entry:
The decision. A single sentence describing what you decided. Precision counts: “We’re going to rewrite the indexer service in Rust” is actionable; “We’re going to improve performance” is not.
The alternatives considered. What were the other options on the table? Listing them forces you to acknowledge that you had choices, because in hindsight the path you chose always feels inevitable.
The rationale. Why this option over the others. This is the most valuable part of the journal, because it captures the reasoning you’ll forget within weeks. Be specific: “We chose Rust because the current Python implementation can’t handle the throughput we’ll need after the Nakamoto upgrade, and our staff engineer has deep Rust experience” is useful. “Rust is faster” is not.
Your confidence level. A percentage or a simple high/medium/low. This is the entry that makes calibration possible, so resist the urge to be vague. Pin yourself to a number.
The expected outcome. What does success look like, and by when? “The rewrite ships in Q3 and reduces p99 latency by 40%” gives your future self something concrete to evaluate against.
A review date. When should you come back and compare prediction to reality? I typically set this for six months out, albeit some decisions warrant a shorter or longer horizon.
A decision journal creates a written record your future self cannot revise. That constraint, simple as it sounds, is what makes calibration possible.
The Quarterly Review
The journal entries are half the practice. The other half is the review, where the real learning happens.
Every quarter, I block two hours and pull up decisions from six to twelve months prior. For each one, I compare what I predicted against what actually happened, and I classify the result along two axes: was the outcome right or wrong, and was the underlying reasoning sound or flawed? That four-way classification matters, because a decision can succeed through luck and fail despite rigor, and confusing the two is how leaders develop false confidence or unwarranted self-doubt.
The patterns that emerge are specific and actionable. You might discover that your confidence on technical architecture calls is well-calibrated, whilst your confidence on hiring decisions runs consistently hot. That asymmetry is invisible without the journal. Once you see it, you can adjust: seek more input on hiring, trust your instincts more on architecture, or better yet, delegate more hiring authority to the people whose calibration in that domain is sharper than yours.
Objections
People raise the same objections every time, so let me address them directly.
“I don’t have time.” Each entry takes five minutes. The quarterly review takes two hours, four times per year. If your judgment as a CTO isn’t worth eight hours of annual calibration, I’d question what is worth your time.
“I’ll forget to do it.” Attach it to an existing ritual. I write my journal entry immediately after making the call, which usually means at the end of the meeting where the decision was made. The trigger is the decision itself, and the overhead is negligible because the reasoning is still fresh.
“What if someone reads it and uses it against me?” Keep it private. This is a tool for your own calibration, and it only works if you’re honest. I’ve never shared my raw journal entries with anyone, albeit I frequently share the patterns I’ve extracted from reviews.
“Isn’t the outcome evaluation still subjective?” Yes. Six months later, you’re still dealing with confounding variables and attribution challenges. The journal doesn’t eliminate those problems; it gives you a structured anchor point that’s less susceptible to narrative revision than pure memory. Better signal in a noisy system is still better signal.
The Institutional Complement
In my CTO Operating System, one of the core principles is “document the decision and the rationale.” The decision journal is the personal, private complement to that organizational practice. The operating system ensures that the team can reconstruct why a choice was made six months later; the journal ensures that you, personally, can reconstruct what you believed and how well that belief held up. One serves the institution, the other serves your own growth as a decision-maker, and neither fully substitutes for the other.
The operating system documents decisions for the team. The journal documents your judgment for yourself. Both are necessary; neither alone is sufficient.
Start Today
Open a document and write down what you decided and why, then assign yourself a confidence level and set a calendar reminder to review it in six months. The practice is brutally simple, and that simplicity is a feature. The hardest part is the discipline to be honest with yourself about your confidence, and the patience to let the compounding do its work.
Your instincts are better than you fear and worse than you hope. The journal will tell you exactly where each is true.