You Aren't Planning Enough
📘 Book Review: How Big Things Get Done
Planning requires thinking - and creative, critical, careful thinking is slow.
Megaprojects1 are large-scale infrastructure projects with budgets of $1b or more. They matter not simply because the ‘products’ that they create have far-reaching impacts on our societies, but because they are a perfect example of how we express - or fail to express - our most sophisticated technical competencies and complex organizational practices. In theory, they are the grandest physical, social, and technical achievements of our civilization. In practice, however, the track record for megaprojects is underwhelming.
Bent Flyvbjerg is the world’s foremost academic researching the success and failure of megaprojects2. According to the data that he and his team have collected, 92 percent of all such projects will run over budget, over time, or both - to say nothing of whether they achieve the benefits they were originally intended for. This implies that when we try to operate at this scale our reach, almost inevitably, exceeds our grasp.
Even on small projects though, while the results may not be as catastrophic, we do not fare much better. The data isn’t as good but it is certainly the case that the majority of home renovation projects will suffer from meaningful cost overruns and most of us are familiar with team projects at school or work that drag on well past their due date. Is there anything we can do to try and avoid these issues?
How Big Things Get Done which Flyvbjerg co-authored with Dan Gardner in 2023 tries to answer this question. The book describes a set of patterns shared by all successful projects. Studying these patterns seems like a good place to start to improve our situation. Luckily, the advice can be summed up in one word: Planning.
Today, many who work at startups or tech companies, and even some that work in large corporates or government infrastructure departments, have become suspicious of planning. They believe that success in large, innovative projects relies primarily on a kind of tinkering and creative problem solving. They say that “necessity is the mother of invention”, they should “move fast and break things”, “pivot” often, and be “agile”. From this vantage point the future of any project is generally unknowable and the best approach to discover the path forward is to respond to the immediate problems in front of us with the appropriate effort and creativity.
How Big Things Get Done calls this approach “Think fast, act slow” and believes it is the primary cause of our project woes. If we run quickly at a solution to a problem without a proper understanding of what implementing that solution entails we will inevitably come across unforeseen obstacles that will delay our progress. As these unknowns become known they are liable to sap our momentum and slow us down. The longer the project is delayed as a result of this the more other problems are likely to pile up.
You can imagine constructing a new house. It is discovered that the soil on the lot is unsuitable for the type of concrete that was originally poured for the foundations. This means you need to go back and dig up more area to relay that foundation. However, the contractor responsible for digging has now gone on holiday. Once they return a couple weeks later summer is in full swing and the ground has become too dry to dig without risk of later movement and settling. So, you need to wait a few extra days for rainfall. By the time you have recovered from one problem the cascade of additional issues may have left your original solution impractical, infeasible, too expensive, or even unhelpful for achieving your original goals. “Projects that fail tend to drag on, while those that succeed zip along and finish”3.
Instead, the book argues, we should cultivate an approach of “Think slow, act fast”. If we remain focussed on our aims and take the time to fully map how we will achieve them during the planning stage - while it is cheap and easy to make changes - then we can maximize our chances of completing the right work fast and on budget. During a project it is impossible to avoid some problems coming up, you can never create a detailed enough plan to catch everything, and it would be a fools errand to try. But, with proper planning you can tame a project’s worst risks and create confidence that you have a good approach. Doing so increases the speed of the project and reduces the potential complexity of the problems that need to be solved along the way.
Good planning explores, imagines, analyzes, tests, and iterates. That takes time. Thus, slow is a consequence of doing planning right, not a cause. The cause of good planning is the range and depth of the questions it asks and the imagination and the rigor of the answers it delivers.
To start, this approach means remaining focussed on the original reason for the project. A project’s stakeholders, and sometimes even each individual stakeholder, will have multiple conflicting aims at the beginning of a project. At a minimum these need to be resolved. Yet, even with a clearly defined North star, you will still run the risk of being distracted from this goal as you develop the details of the plan .
The most common culprit for this is becoming solution oriented rather than remaining problem oriented. Once you have a solution in hand, especially if at first it sees like a great answer, you can start to make excuses as you uncover issues with the approach you’ve chosen. If this continues and you still hold fast to the solution you chose you will inevitably either water down your aims to accept a less good outcome, or change them altogether in service of something that originally had only been chosen as a tool.
If the only tool you have is a hammer, it is tempting to treat everything as if it were a nail.
Abraham Maslow4
To avoid this you need to always keep the goal in mind. As Steven Covey says you have to “keep the main thing the main thing”. This means that when you have a solution you must continually test whether it will really get you where you want to go. A particular tactic How Big Things Get Done recommends here is the Amazon practice of working backwards. Before you commit to a solution, as part of planning, write a press release or imagine a launch party as if you had chosen that solution. Is it the outcome you were hoping for?
With clarity on where you will end up once the project is complete you now need to make sure that the implementation of that solution is not going to run into any catastrophic problems along the way. The best way to do this is to iteratively create models of the project with increasing levels of scope and resoloution. The book calls this “Pixar Planning”.
One of Pixar’s unique qualities as a movie production house is that they spend significantly more time planning a movie than their peers. First, a director is allowed to take months to come up with an idea for a movie that seems promising. This can be one line as simple as “A French rat loves to cook”. This idea is then turned into a 12 page outline to show how it could become a movie. After several iterations with feedback this outline is turned into a 120 page script. Again, several iterations will happen. At this point the script is turned into a storyboard that is photographed, and turned into a video with voice actors reading the lines. This generates a full simulation of the final film in miniature and it again undergoes the a process of intense scrutiny and iteration before finally being ready for full scale animation and production. By the time the final film is greenlit it will have undergone this storyboarding process an average of 8 times.
At each stage of this process the film’s creators are developing a more accurate model of what the final film will be. It is an expensive and slow process, but, because it is still in draft form, nowhere near as costly or slow as making changes to the film once it is already in the final production phase.
How Big Things Get Done’s admiration for this method helps explain the gap between the approach that Bent Flyvbjerg calls for and the kind of tinkerer, creative culture behind the sentiments mentioned at the beginning of this piece. In particular, it may seem that this emphasis on planning is dismissive of the Lean Startup methodology that informs ideas like pivoting, and failing fast, and has taken root as a core process tool in the creation of new tech companies. In truth, the books thinks that the Lean Startup is an excellent methodology. But, it needs a reframing.
Under Lean Startup builders are instructed to focus on running experiments fast with real data and then iterating to get to a working product and company. This means having an idea, rapidly building a low quality (i.e. buggy) Minimum Viable Product (MVP) to illustrate that idea, putting it in front of users to get feedback and then, if the MVP doesn’t work, pivoting to a new idea and building a new MVP. The theory behind the method is that through emphasizing feedback and being open to go in whatever direction makes the most sense for your target market you can bootstrap to something that works for users. Seemingly the opposite of a plan.
For Bent Flyvbjerg though this precisely is the work of planning. You have specified your goals with precision: “Make something people want” then you have set about creating ever more refined models (MVPs) for how to reach that goal. At this stage mistakes are still low-cost and recoverable. In other words you have created a great plan. The execution of the plan when the company “scales” is a very different matter that comes later.
Planning is doing: Try something, see if it works, and try something else in light of what you’ve learned. Planning is iteration and learning before you deliver at full scale, with careful, demanding, extensive testing producing a plan that increases the odds of delivery going smoothly and swiftly.
Unfortunately, there is a limit to models. By definition, a model is a low-dimensional representation of the full project. 3D-printing can never replicate welded steel. Pseudocode can never replicate a production system. This means that once the project is underway there will always be new issues that come up that a model alone can’t help you plan for.
The best way to prepare for these kinds of problems, that can’t be captured explicitly in a model, is with expertise. When someone works on a project they develop a body of practical, tacit knowledge that they can bring with them to similar future projects. This is why architects are often considered to do their best work in the last decade of their careers rather than their first, the weight of experience from seeing so many buildings be constructed simply outweighs any kind of brilliance in the theoretical stage before those buildings come to life. All new projects have some existing analogous projects that have been previously completed. So, bringing on people who worked on these completed projects to help with the new one is a way to ensure the plan has been vetted not just by ideas, models, and theories but by experience.
Precisely because it isn’t something that can be acquired in the planning process this experience is hard to fathom. If we do not have it ourselves the best we can do is gesture at the fact that someone who has that experience should be able to make sure the project can be a success. Yet, it is invaluable to have around.
We do not make the most of experience because we do not appreciate how deeply experience can enrich judgement and improve project planning and leadership.
The final ingredient for a successful plan is standardization and more specifically creating modular components. This does two things to transform the project’s effectiveness. First, it allows the project to take full advantage of the learning curves that come from creating a smaller set of components at a larger scale.
Second, it means that the project can be flexible to changing demands. If there is a small fixed number of potential components then all it takes to change the project scope is to change the configuration of those components. The canonical example of this is solar power:
[Solar power is] born modular, with the solar cell as the basic building block.In a factory, put multiple solar cells together onto a panel. Ship and install the panel. Install another, and wire them together. Add another panel. Add another until you have an array. Keep adding arrays until you are generating as much electricity as you like. Even giant solar farms consist of little more than that. Solar power is the king of modularity. It is also the lowers-risk project type of any I’ve tested in terms of cost and schedule.
The alternative to standardization, and modularity, is one-off and unique. Which of course brings all kinds of dangers. One-off means non standard and non modular, beyond this though the more unique a project is the less talent and experience you can take advantage of in planning, the less confident you will be in creating models that reflect the final reality, and the harder it will be to remain tied to your aim. If you want your project to succeed uniqueness is the enemy.
How Big Things Get Done is not a romantic book. There is little respect for the passionate, creative, extraordinary, or heroic entrepreneur who will break down walls and challenge norms to get things done. However, it is sentimental in the way that all great works of economics are. It imagines a better world that could be achieved through great care, discipline and thoughtful planning. Another way of saying this is that the approach Bent Flyvbjerg advocates for is a kind of definite pessimism in opposition to the indefinite optimism that characterizes most projects. It strikes me as very Danish5.
Ultimately, it is a work of popularization. So, if you are looking for specific advice on how to actually build a project it would make more sense to look at some of Bent’s more detailed academic work or, even better, case studies on projects that look similar to the one you want to create. However it is an excellent entry point for thinking about the fundamentals of project design and delivery and offers a posture that we could all learn from. I plan to re-read it
Emphasize planning, work backwards from what you want to achieve, create iterative models of the final project, focus on amassing as much relevant experience as you can, and seek modularity wherever possible. If we are looking to become more competent in getting big things done these principles are not a bad place to start.
Takeaways
Plan more. If you want a project to succeed you should spend much more time planning than you think you need. The more you can de-risk up front the more likely you will achieve what you set out to on-time, on-budget , and with the desired outcome. “Over”-planning is possible but unlikely.
Plan wisely. You have to fight with every project to keep the main thing the main thing. As solutions are proposed, especially if at first they seem like they fit your needs, it is easy to become solution oriented rather than remaining problem oriented. When that happens even if you deliver the solution as expected (unlikely) you run the risk of building the wrong thing.
Planning is doing. To create a solid plan you need to run experiments and get feedback. Model the entire project from start to finish in a rough form and see what your solution looks like. If you can’t build such a model in the real world then find case studies that map as closely as possible to the project you are aiming to complete.
Value experience properly. When it comes to completing projects on-time, on-budget and with the expected results there is no substitute for working with people who have done it before.
Standardise. The best way to give yourself an edge in bringing down complexities, costs, and risks is to standardise the elements of your project. Create modular components to take advantage of learning curves and the ability to scale up and down as needed.
Bent Flyvbjerg’s definition: “Megaprojects are large-scale, complex ventures that typically cost US$1 billion or more, take many years to develop and build, involve multiple public and private stakeholders, are transformational, and impact millions of people.” Source
He is also a much sought after project manager having consulted on multiple multi-billion dollar projects for the UK, Sweden, Hong Kong, Denmark and more as well as having other interests as an academic.
Unless otherwise noted otherwise all quotes come from How Big Things Get Done
I don’t know a ton about Denmark, I’ve never visited - though I would love to - but this certainly fits with my sense from reading Kierkegaard.
This was fantastic, and very timely for me, thank you!
"Planning is doing" may be the most under appreciated point.
I'm privy to a group involved with a US semiconductor initiative working on the order of $B. My current takeaway from the initiative is that they're trying to build the framework to build the thing before building the thing once - a common failure mode.
So they aren't planning by doing, they're planning by planning. Building one thing first before scaling it up would be a huge de-risk, but it probably doesn't seem like enough work to justify the $.
In software, planning by doing might also be similar to the concept of "tracer bullets" mentioned in The Pragmatic Programmer book.