Megaprojects and getting big things done with Bent Flyvbjerg

Disclaimer We'd like to advise that the following program may contain real news, occasional philosophy, and ideas that may offend some listeners.

Bent Flyvbjerg You know, 'think slow, act fast', that's the rhythm of a successful project. That's what we see at Pixar. That's what we saw at the Empire State Building. That's what we see in the many other successes that we include in the book.

Sandra Bent Flyvbjerg is an expert on some of the world's most epic projects. From the Olympic Games and airport terminals, all the way to IT projects and private homes. He has consulted on over 100 projects costing over $1 billion and has even been knighted by the Queen of Denmark.

Kai And Bent's most well-known for his work on megaprojects. But really, his aim is to help all project managers, from industrial builders to home renovators, to bring their project in on time, on budget, and with their objectives achieved.

Sandra Bent's latest book is called 'How Big Things Get Done', which he co-authored with Dan Gardner. And Bent is the first BT Professor at Oxford University and VKR Professor at the IT University of Copenhagen.

Kai So Sandra, this week we have highlights of your interview with Professor Flyvbjerg, which we recorded as part of an event in March this year. And you started talking to him about the Sydney Opera House and how it got built, which given his Danish heritage is a project dear to his heart and mind.

Bent Flyvbjerg It's really a story made for a Greek tragedy, what happened with the Sydney Opera House, unfortunately. It was a disaster, you know, in terms of execution, and in terms of the career of the architect which was destroyed.

Kai And this landmark which puts Sydney on the world map and is famous for its beautiful architecture, as we all know, is also famous for its 1,400% cost overrun.

Intro From The University of Sydney Business School, this is Sydney Business Insights, an initiative that explores the future of business. And you're listening to The Future, This Week, where Sandra Peter and Kai Riemer sit down every week to rethink trends in technology and business.

Sandra Since we're hosting this from the University of Sydney, let me start with your views on what we can all learn about better project management from the story of the Sydney Opera House.

Bent Flyvbjerg Well, first, let me start by saying that it's one of my favourite buildings in the world, probably the Sydney Opera House and the Guggenheim Museum in Bilbao. They are my two favourite buildings. And I don't think I'm alone in this, there's general consent that they might be the two greatest buildings of the last 100 years. So that's really something Yeah, it's just a magnificent building. I've been there several times. And I always feel uplifted, you know, and I think this is what great art does, no matter what kind of art, it lifts you up, and the Sydney Opera House certainly lifts me up when I visit it. And I know many other people have the same experience. But you know, it's really a story made for a Greek tragedy, what happened with the Sydney Opera House, unfortunately. I mean, maybe many people don't know this, or have forgotten this, and so on. But it was a disaster, you know, in terms of execution. And in terms of the career of the architect, which was destroyed, you know. Name me another building that the architect has made - there is no other major building by this architect.

Sandra Jørn Utzon, who's the...

Bent Flyvbjerg Jørn Utzon, yeah, so...

Sandra He was fairly young when he did this.

Bent Flyvbjerg Yeah, he was 36 when he won the competition, and that's very young for an architect. And I've talked to more than 1000 people, probably a couple of 1000 people about the Sydney Opera House and asked them the question, 'Do you know who the architect is?' And very, very few people know who the architect is. And even if they know, you know, then I ask, 'Okay, second question. Can you name me another building by that architect?' And of course, nobody can, because there are no other buildings, except you know, the connoisseurs know, there are a few small buildings in Denmark, and also an assembly in Kuwait. But this is the main thing to learn, you know, is that you can build things so that you destroy the careers of the architects. That's not okay, to be so inconsiderate in the way you do things. So we need to have better project implementation than that. And one thing to learn from the Sydney Opera House is you can't take 14, 16 years to do a project. It's just way too long. That's one thing, regarding time, you know, that you need to keep it short when you're doing projects. If you want to do something that's successful, it has to be short. Another thing is that you can't start until you know what you're doing, which is what they did with the Sydney Opera House. They actually had to dynamite out parts of the Opera House while they were building it because they built the wrong thing because they didn't have the final drawings when they started doing the building. Not a good idea to start building before you know what you're building, obviously. So we talk about this in the book, we call it 'think slow, act fast'. The Sydney Opera House, they did the exact opposite. They thought fast, they really got going very, very quickly, because this was a mandate from the government that you had to start building now because they were afraid that the decision might be overturned to build the Opera House by a new government, so they wanted to build as much as possible as quickly as possible to get to a point no return, which is quite common, but it's a really bad idea. So they thought fast, and then they were forced to act slow because so many problems occurred during construction. And I think we have to be really grateful that the part that Jørn Utzon, the Danish architect, got to do was the outside. So he actually didn't do the inside because he left the project in the middle of the project. He was forced out. The New South Wales government simply stopped paying his salary, you know, and he went back to...

Sandra That's one way to do it...

Bent Flyvbjerg Yeah, that's one way to fire somebody. So he went back to Denmark. But before he went back, he had actually overseen the design of the shells and putting the sails up. I remember the first time I came to the Opera House; I was shocked to walk from the outside to the inside. The inside looked like a 1970s discotheque, you know, and obviously, that wasn't designed by Jørn Utzon. And that was designed by some architect, some local architect that took over after he left. And basically, that architect destroyed his career by taking over this project by Jørn Utzon. And so another architect’s career was destroyed by the Sydney Opera House, because of the mismanagement of the project. So yeah, I'm delighted that we have this wonderful building, obviously, as anybody should be. But boy, could that have been done differently? And much better.

Sandra You've built one of the biggest databases in terms of megaprojects. You've mentioned two very famous ones here, the Opera House and the Guggenheim Museum. But much of your research has explored these megaprojects, I mean, 16,000 megaprojects, and you tell us what the iron law is, how does it play out in the kinds of projects that you discuss in the book.

Bent Flyvbjerg So first, it's not 16,000 megaprojects, it's 16,000 projects, of which most of them are quite large projects. But in order to understand whether size actually makes a difference to the outcome of projects, we need to study both small, medium size, and large projects. So we have all sizes in the database. The iron law is very simple, it says 'over budget, over time, under benefits over and over again', it's a statistical law. And it holds with a very high level of statistical significance, actually unusually high for human behaviour. And it means that if you're planning on building a project, then you have a very, very high likelihood, the odds are that you will be over budget that you will be over scheduled. So you will have to spend more money than you thought, you will take longer than you thought. And you won't deliver the benefits that you thought. That's just the odds. It's like going to a casino. These are the base rates, you know, and it's really important to face up to that, especially if you want to change. If you play cards, if you're a poker player, you need to know what the odds are, if you want to have a chance of winning, right? It's exactly the same here. You need to know that the odds are actually against you, like in any game you're playing, in order to win at the game of building big projects. And if you don't understand this, you don't have a chance. You're like a babe in the woods, and wolves will take you.

Sandra But it's a bit like in poker as well. Everyone thinks, yeah, yeah, that's true, but it won't apply to me. I'm the lucky one, the one who will have the lucky winning hand at poker and so on.

Bent Flyvbjerg The professional poker players don't think like that. But you're right, the rookies do.

Sandra So what factors contribute to the iron law, what makes the iron law.

Bent Flyvbjerg So it depends on what level you look at it. In the book, we look at it at what we call root causes. So we tried to find the most fundamental causes of this. And at the root cause level, we find two types of causes. One is psychology, and the other is power. And on the side of psychology, it's cognitive biases, and you had Danny Kahneman as a guest. So he's the pre-eminent scholar on cognitive bias, optimism bias, anchoring, overconfidence bias, availability bias, et cetera, et cetera. There are lots of cognitive biases that are very well documented. For instance, optimism bias is one of the most pervasive biases, and a bias that most people know and understand, that we are optimistic about things. So if you're optimistic about the budget on a project, you think it's going to cost less than it actually costs, right. That's optimism. You think it's going to be cheaper. If that's the way you plan the project, develop the business case, then just like the law of gravity, your optimism is going to boomerang around and hit you as a cost overrun, right? If you are optimistic about the budget, you're going to have a cost overrun. Same with the schedule, if you are optimistic about the schedule, you're going to have a schedule overrun. Same with benefits, if you are optimistic about the benefits, you're going to have benefits under run. So you're going to under deliver on your benefits. So that's how optimism and other cognitive biases as root causes influence what is happening on projects. That's the psychology side, the power side we have what we call strategic misrepresentation. And on the power side, things are deliberate. So the psychology side, things are not deliberate. People don't do that consciously. So we are not optimistic consciously. Actually, deliberate optimism is an oxymoron. If it's deliberate. If you are deliberately optimistic, it's not optimism, then it's something else. It's probably strategic misrepresentation. And strategic misrepresentation is like you lowball the project in order to make it look good on paper. So politically, you decide I'm going to make my project look good on paper. How do you do that? Well, that's easy. Anybody can do it, you know, you lowball the cost as the Americans call it, and you lowball the schedule, and you overestimate the benefits, then you look really pretty on paper. And you have a good chance if you are evaluated, assessed against other projects, that your project is going to come out on top, because you falsely misrepresented the cost, the schedule, and the benefits. Strategic misrepresentation's power actually works like psychology, the main difference is that one is unconscious, and the other is conscious. But they both contribute to the cost underestimation, and then as a consequence, cost overrun. And the same with time and benefits. Those are the two main root causes. Of course, if you look at it at sort of the more superficial level, you'll hear people say, 'Well, we had cost overrun, because the weather was bad when we were on the construction site. So we had to take time off.' Or, 'When we started digging, we found things in the ground that we hadn't expected, you know. And because we found those things, the project took longer. We found, for instance, an archaeological find, you know, so we had to stop the project for some time. And then do the archaeological dig, like the law says we must, and so on.' And that's also true, you know, these things happen, for sure. Or, you know, 'The price of steel went up. So the price of the project went up.' We don't think that these are root causes, because if you think about it, we dig in the ground all the time. And if you keep underestimating what you find in the ground, that's actually optimism, you were optimistic about what you were going to find in the ground. Or rather, you were optimistic that you weren't going to have as many problems as you actually encountered. Right? That's optimism. That's not because the geology or whatever tripped you up - you trip yourself up. That's why we say, 'your biggest risk is you'. That's one of the sayings in the book, and one of the, what we call 'heuristics'. This is a good example of that, that it is actually not the objective risks out there that is the problem. It is our interpretation of risks. And because of our cognitive biases, we misinterpret risk all the time, like, easily illustrated with optimism, like I just did.

Sandra So psychology and power as the root causes. But I do also want to take you to some of the things that lead to success. Before I take you there, I want to hear a bit about projects that were successful. And one of the most interesting examples in your book is the Empire State Building. And I mean, just the numbers to me are staggering. This was completed 17% under budget, right on schedule, the whole thing from kind of design to delivery in under two years, for many of us from Sydney that just seems completely impossible to do today. Tell us a bit about what made the Empire State Building, and why was it so successful?

Bent Flyvbjerg Yeah, I'm glad that you are asking about success, because actually, we really made a point of including a lot of success stories in the book, because I was getting tired of you know, because there are so many projects that have gone bad and you study projects, and all of a sudden, I began to get the reputation that hey, he's the scholar of doom, you know, on projects. And that's not fair. And that's not what I want to do. I actually think the successes, of course, are at least as, and I would say, more important than the failures. And the Empire State Building is a great example. And it shows us that we have been able to do this, you know, always, we can actually do successful projects. And we were able to do it 100 years ago, we were able to do it 200 years ago, and we are able to do it today. So it's not like we don't know how to do successful projects. It's just that we don't do it. And that's very interesting. Why don't we do it when some people know how to do it? The Empire State Building was as successful as it was because the people who built it really knew what they were doing. Really, really good people. And they were thinking slow, not in the sense that they were taking years to think out how to build the project, they actually thought it out within one year, but they used that time really well. And it was a long time compared to the construction period. So it took them one and a half years to build the building. And it took them a little less than a year to design the building. So they used a fair amount of time in order to get it right. And this is the right way to do it. And they did that. Not only that, they had actually already built a smaller skyscraper almost exactly like the Empire State Building. Not a lot of people know this, but they built it for the Reynolds Company. So this was a famous tobacco company in the US, and they build a headquarters in another town. And it was designed pretty much like the Empire State Building. So the architect and the builders had already built one skyscraper like this. Now they just had to do it one more time, and taller. So they had experience. That's another thing we emphasise in the book, you need experience. Also, they used a modular design, they had completely designed the building from top to bottom and they did it in a way where the majority of floors were exactly like other floors. So they actually said, you know, to illustrate what they were doing, we actually did not build a skyscraper, we just built the same floor 100 plus times and then we stacked them on top of each other. That's why the building is so tall.

Sandra So that would mean that they just get better and faster every time they build it again.

Bent Flyvbjerg This is why we emphasise replication in the book, you need to do things in a way where you replicate something over and over. Because when you do that, you get what we call 'positive learning curves'. It means that you get better and better, you get faster, and you get cheaper at what you're doing. And that's exactly what happened on the Empire State Building. They actually built more than one floor a week, substantially more when they really hit their stride. So in the beginning, it was a bit slower, but then they got better and better as they replicated these floors. And they got up to a level of where they were building several floors per week. And then it tapered off again towards the top.

Sandra So, Empire State Building, really great success story. One of the other really interesting examples in your book is Heathrow Airport, Terminal Five. And I think everyone is kind of familiar with what disasters airport projects normally are. They're very complex, they're very difficult to deliver. And you might expect from something like Heathrow for this to be a complete disaster. However, this wasn't really the case with Terminal Five. And that was quite a successful project. And I think since Sydney is currently planning to build its second long overdue airport, it's worth having a conversation about Heathrow's Terminal Five. What made that successful?

Bent Flyvbjerg Basically, the owner, which was BAA at the time, decided that we can't build an airport the way airports are normally built. The first thing they did, actually, again, using 'thinking slow, act fast', during their thinking slow period, they went out and studied other airports. And they came to the conclusion after having started other airports that if they built this airport the way other airports are built, they were going to go one billion pounds over budget, and they were going to go, like, one and a half years over schedule or whatever. That's not the exact number, but something like that. And they decided we're not going to do that for a very simple reason - we can't afford it - you know, our company will go bankrupt if we do it like that. So we need to find another way. And the other way was for BAA to take on the risk of building Terminal Five. So instead of trying, as you usually do writing contracts, where the risk is allocated to the builders, they said, 'No, we're not going to do that we are going to use a partnership model where we as the owner, we keep the risk. And then we are partnering with the builders with just one goal. And that is to deliver this airport to the budget and to the schedule and to the high quality that we have stipulated in the design.' And they succeeded in this. And what they got from their partnering model was that they avoided all the conflicts that you typically find on a construction site where immediately I mean, sometimes even before you even start a construction, people are suing each other. You know, 'no, no, this is not my risk, this is your risk. Because if I have to take the risk, then you know, that's going to cost me money. And I don't want that because I want to make money on this project. So this is actually your risk.' And then the other party says 'no, no, no, it's your risk.' And then you have a conflict immediately. And you need mediation as a minimum. And sometimes it actually turns into real full-out conflict. It's very common. That's why, you know, people often talk about the construction industry as a very antagonistic industry where people are suing each other right and left, you know, because they're really afraid of the cost of all the risks that they're taking on, for good reason. Because these risks can kill you. So if I have to point out just one thing, that's the main thing, but there are lots of other things, you know, like there was a real team building spirit in the leadership and actually from the top to the floor, in the whole organisation. Everybody felt like a team. And this is something that the leadership did everything to cultivate, to take good care of the people who worked on the site to make sure that things were safe, and so on. Yeah, they just did the right things, because they were very experienced, they had tried this before they knew what it takes.

Sandra I think you describe that in your book about make friends and keep them friendly, which is not something that readily comes to mind when you think really big projects, and especially construction industry is not something where people normally make friends and keep them friendly. You mentioned a few times 'think slow, act fast'. And I want you to reflect a little bit on that, because you often return to this example from Pixar, in your book. Can you talk a bit about their planning process and the type of work that goes into their projects, from where the 'think fast, act slow' comes from.

Bent Flyvbjerg Sure. Yeah, Pixar captured my attention because well, they're great movies, but also because they have done 20 plus blockbusters in a row. So they only do films that are extremely, not just successful, but extremely successful by Hollywood standards. And if you read film history and so on, or just the news, actually and I lived in Los Angeles and everybody talks about the film industry there. I actually went to UCLA as a student, and therefore this is just a habit you know, following the film industry a little bit, and I knew that film is, like, hit and miss, you know, you win some you lose some, and nobody ever knows beforehand. It's one of the most difficult things to forecast. So all of a sudden, after 100 plus years of Hollywood history, this studio pops up. And they are able to do it again and again and again and again and again. And they just keep doing it. Right. How did that happen? So I got very curious about this. And luckily, the people at Pixar were willing to speak, you know. Also Ed Catmull, has written an excellent book, by the way, it's called 'Creativity, Inc.', that I recommend. Ed Catmull also lectured here at Oxford, and we interviewed him for a long time for the book. And the same with Pete Docter, who's the Creative Director at Pixar - we got to pick their brains. And what we found was that they're very, very keen on this iteration that we talked about before, like the Empire State Building, you know, they build the floor over and over. And they do it actually, before they build it in reality, so they simulate it. Pixar has a process by which they go through eight iterations of a film before they actually start shooting it. So anybody at Pixar who's a director can come up with an idea for film and Pete Docter explained that often it's just an idea, like, he says he gets his ideas in the shower. So it might just be like, 'a girl who lives in her head'.

Sandra Old grumpy guy?

Bent Flyvbjerg Yeah, old grumpy guy, a rat that cooks, you know, a rat that cooks? That sounds crazy. So it's just an idea like that. That's how a film starts. And then the person who has that idea might decide to sit down and write about ten pages about how would you unfold this idea in a synopsis of what a film could be like, that's the first step. And then you circulate that synopsis to the other directors at Pixar, and you get feedback. And people will say, I like this, I don't like this. That feedback is not something that the director who got the idea has to follow, it's an option. So this is all opportunity for the director, you know. You get all this feedback, you can use it if you want, you don't have to, if you don't want. And then you do a third version, and you do a fourth version. And then around that time you start doing storyboards. So you start making images, and it's all a physical thing. And first, only a few dozen. And then eventually, the storyboards develop into you know, 3000 plus storyboards for a typical film, and they start taking pictures of the storyboards, maybe just with your phone, you know, and then you can actually run the pictures after each other. And you get an idea of what will this look like when it becomes a film, you think what kind of music would be nice behind this, you add the music, and you try to add some voiceover, just yourselves, you know, just saying whatever the dialogue is, you put it in there. And eventually going through these eight iterations, you get a more and more complete version of the film. And only when you have that complete version that you are satisfied with - so you've had these seven, eight rounds of feedback, and you've incorporated whatever you thought was right, and now you have a version that you really believe in - that's probably two years in. So this is the thinking slow phase, at Pixar it's about two years. And then they start shooting. And that's the acting fast, and only then do they start shooting, which is on these expensive animation computers that they have. And they hire in the famous Hollywood actors who are doing the voiceovers for the final version of the film. And they hire in expensive composers who are composing the music for the film, and so on. But only in this phase because this phase is very expensive. Once you start shooting, it's really expensive.

Sandra Maybe a couple of years with those iterations until you bring in the real actors.

Bent Flyvbjerg Yeah, then you shoot the film in a relatively short period of time. But because of all the preparation you did, you can do it in that short period of time. And the preparation is dirt cheap compared to the actual shooting. And that's why you want the cost to be on that side, and not on the shooting side.

Sandra So 'think slow, act fast,' and also a fantastic nod to Danny Kahneman's work there.

Bent Flyvbjerg Exactly. So his book is all about thinking fast, and the kinds of problems that gets us into, and we should think slow instead. So we're taking the think slow part and saying that's what you need to do. Don't think fast, think slow. But Danny is talking mostly about thinking. So we also talk about action. So that's why we put this second part on, acting. And it's really important to see the relationship between the two, and we see it as the rhythm, you know, 'think slow, act fast', that's the rhythm of a successful project. That's what we see at Pixar. That's what we saw at the Empire State Building. That's what we see in the many other successes that we include in the book.

Sandra I think this is a perfect moment to start going to audience questions. We really always want to bring the community along. So I want to make sure we have enough time for that. But our first question comes from Professor Percy Allan from the Institute for Public Policy and Governance at UTS, across the road. And the question is, 'Are large public sector projects more prone to failure than large private sector ones? Also, are the key reasons for failure in each different or the same?' And there's a related question to that, that I might bring up as well, which asks, 'Having looked at so many different projects, which are the best and which are the worst for on time and on budget delivery?' So looking at large private sector versus public sector projects.

Bent Flyvbjerg Yeah, that's a good question, and I'm actually working on a paper right now about the difference between public and private IT projects. We're very interested in IT at the moment. And this is one of our big studies in IT. So far, the result indicates that private projects are indeed performing better than public projects. But this is not what I would take away as the main message. The main message is that both are performing bad. It's really important, you know, to realise that both public sector projects and private sector projects are underperforming. However, it looks like public sector projects are underperforming somewhat more than private sector projects. But I will say that's the secondary message, the primary message is that both types of projects are underperforming. Second question was, what are the reasons for that? We actually don't know. And nobody knows. Nobody has studied this at a level where in scholarly terms can say that we now know what the reasons are for this difference. We can speculate. I mean, in a way, if you go back to neoclassical economic theory, it's already there, where you know, the public sector is there to do projects that the private sector will not do, because they are too risky, you know, or there's not enough profit in them, but they still need to be done for society. So maybe the public sector is taking on such projects. And maybe such projects are more difficult and more complex and more risky than the private sector projects. And that alone would explain this. But this is theory, but it's a very central economic theory that you will find in Economics 101, you know, but I haven't seen any study of projects that actually, without reasonable doubt, documents that this is the theory that explains the difference. But we are doing research on this. And we are going to be studying more project types for this difference, including transport infrastructure. Third question was, what are the best and what are the worst projects? There's a table as an appendix in the book where you can see this, and you will see that the worst performing projects are - the stories of nuclear waste is the worst performing project, the Olympic Games, and I understand that Brisbane has decided to host the Olympic Games, I'm saying 'good luck!' Look at the book, read about how the Olympics are performing, and then figure out how you can do it different, and show the world that the Olympics can be delivered on budget - on time is not a problem for the Olympics, obviously, because you don't get to mess with the opening dates. But on budget is a real problem. Actually, we have data going back to 1960. Not a single Olympics has been on budget. So Brisbane has a huge opportunity here. And I'm encouraging Brisbane, take this opportunity to become world famous by being the first city that actually delivers the Olympics on budget. And I promise you, you will be in my next book, if you do that.

Sandra We have people from Brisbane on the call, so...

Bent Flyvbjerg Okay, so maybe we could have a bit of fun with this question. Nuclear power is bad, IT is bad, I already mentioned that, hydroelectric dams. So these are the worst performing projects. At the other end, you have... the best performing projects are actually solar farms. So solar farms, because they are the most repetitive project, if you think about it, it's a solar cell, put on a panel, and you put the panels in an array, you have many arrays, you have a farm, and solar farms are the best performing type of megaproject, you can build a billion-dollar solar farm really fast, really easy. And you can keep it on budget. Next best performing is wind power. And third best performing is conventional power plants, oil and gas and coal. And then energy transmission is also very well performing. And those are the four top performers at the good end of the curve. And then we have like 20 projects in between, you know, like rail projects, and skyscrapers and so on that are on the middle ground and the middle ground is still you know, like buildings have a 60% cost overrun on average. That's the middle ground. And when we talk about the others, we're talking about very, very high-cost overruns.

Sandra Let me then ask you about another kind of megaproject because we've also had a very interesting question around something that's quite current in Australia, you might or might not know that Australia has committed $368 billion to the AUKUS nuclear submarine project with the US and the UK. We're aiming to deliver eight submarines by mid 2050. What advice would you give to land this on time and on budget?

Bent Flyvbjerg Well, defence is actually one of the bad sectors also. It's not quite as bad as the Olympics, but it's pretty bad. So again, I would say, read the book, you need to follow the principles in the book if you want to be able to succeed with something like that. And like I said, defence products are notoriously underperforming. There's something about defence and the defence industry. You know, they have a bunch of bad habits, let's put it that way. And they keep being allowed to reproduce the bad habits, maybe in a sense, because they are more protected than other parts of project delivery, that it's government and it's a very high priority, and many projects are one off, you only do a few of them and so on, so they may be a bit of a different type of animal. In my mind, that doesn't mean that you shouldn't be able to deliver them in an efficient manner.

Sandra Before we get to our final questions, I do want to bring up a few issues that seem to transpire through some of the questions that we have, which is, lessons for kind of startups and innovators or people who have very unique projects where this idea of experience or having repeated something that you've done before, might not quite be as useful.

Bent Flyvbjerg But this is exactly what startups do, you know, they do minimum viable products, right, and then they just ship it, and then they see what happens. And then they do it again, they ship it and see what happens, they do it again. So this is repetition. We also cover this in the book. So we actually don't see any difference between the way you do a small startup successfully, and the way you do a big project successfully. The main difference is that you have to be extremely careful if you are doing something that is not reversible, or that is unsafe somehow. So obviously, you could not build the Empire State Building as a minimum viable product and then say, you know, 'If we don't like it, we just take it down again and build another one'. That doesn't work for some projects. So that's an irreversible decision. So that needs to be treated differently from something that's reversible. So reversible decisions, it's very easy to experiment with, and actually do the replication. And don't worry too much about how good your product is, because you can always reverse the decision. One exception is if you're doing something that has safety issues. So for instance, if you're doing something in the pharma sector, it's different from software. Software is quite forgiving, you know, you can make lots of mistakes in software, and unless it's on a plane or rocket or something, it doesn't necessarily have bad consequences. If you're doing medicine, you can't experiment like that. You can't just ship the medicine, see how it does, and then if people die, then you just make a better version of the medicine. This is actually what Theranos, I don't know if you know this company in Silicon Valley, that developed a blood testing device, and the founder, Elizabeth Holmes, has now been sentenced to 11 years in jail, because she used this kind of model, and it just ran away for it. I mean, she ended up doing fraud to cover over things because she shipped too early, basically...

Sandra Bad Blood is a great book.

Bent Flyvbjerg Yeah, exactly. Bad Blood, that book, it's right here on the shelf behind me. Yeah, so that's not a good idea to use that model in something like that, if you're doing something that is related to medicine and health, or something that is related to safety, where safety is a real issue. But other than that, you know, you can do it.

Sandra We've got a couple of interesting questions. One is from Christopher Standen from UNSW, across the other road, and he wants to know how we can prioritise transport infrastructure investment to optimise for social outcomes, things like climate or safety or less sprawl and not individual benefits, like increased speed, for instance.

Bent Flyvbjerg That's easy, you just decide to do it. And that's what you focus on. Instead of emphasising traditional cost benefit analysis, you emphasise a social impact analysis or an environmental impact analysis. And that's actually happening more and more, it's been very slow going for social and environmental concerns to get a preeminent position in projects, in my view. I mean, this is something people started talking about, at least 40 years ago. And it's still lagging, in my view, but it's happening more now than it has ever before. So I think that there's a good chance that these types of concerns will have a stronger position going forward. But it's still way too weak in my view, but that's not because we don't know how to do it. That's because people don't want to do it.

Sandra Have you got particular areas or regions of the world or spots that do it better than others where there might be an ecosystem or cultural influences that make it more prominent?

Bent Flyvbjerg So when you say 'it's' what is 'it's' here?

Sandra Projects that prioritise social outcomes. So, particularly good at prioritising climate or safety or less urban sprawl.

Bent Flyvbjerg So that would be the Netherlands, that would be Denmark, where I'm from. Sweden, Norway, would be like top of the line there. But still, they're not doing nearly enough. They can't rest on the laurels, you know, but they're doing more than others, especially if you look at the way cities are designed, you see social and environmental concern being integrated much more than they were just 10 or 20 years ago. So it's moving in the right direction. I think there's still too little and it's too late. But it's better than if it wasn't there. And there's going to be more in the future. I have no doubt about that.

Sandra So since we're talking about different cultures, there's another question from our audience that, you know, given that psychology and politics are both formed within particular cultural contexts, the question is whether you've come across any cultures or nations that appear to be especially good at managing megaprojects?

Bent Flyvbjerg That's a very good question. And the answer is no. And it's surprising. There is no geography. We studied more than 130 countries. So we have more than 130 countries in our database, and for years now, we have looked, is there a country that knows how to do this so we could learn from them? And the answer, the sad answer is, that there is no such country. There are some countries in periods that have been a little better than others. But again, it's like what I talked about before, it's just that they are less worse, they're less bad, you know, it's not that they get it. So the Netherlands, for instance, are slightly better than other nations at delivering transport infrastructure. The United States, during a period when they were building large dams, they were better than other countries at building large dams. So there are nuances like that. But there's no country where we can say 'they've got it, they know how to do it'. There are countries that used to think that they knew how to do it, like Germany. Germany has had a big reckoning lately, with a lot of projects going wrong, most pre-eminently Berlin Airport. We talked about before, that many airports are not built successfully. And Berlin Airport, is, if I ever saw one that wasn't built successfully, that's it. You know, it's like the poster story for how not to build an airport, unfortunately. And there have been several projects like that in Germany that have given them a bit of an identity crisis. They think that they are much worse than other countries now, and I have to reassure them 'no, you're just getting down to the level of other countries, you're just like everybody else', you know.

Sandra What's there's something in particular, that made Germany come back to the average level, because they did have this aura of being extremely good at managing large engineering projects, construction projects.

Bent Flyvbjerg It might just be there's no cause other than it's regression to the mean. They weren't successful for that long a period, you know. This is like the post war period. And then up to like, 15 years ago, things started to go a bit wrong with some projects. And then there were more and more and then Berlin Airport. And now there's the whole issue with the car industry that's actually traumatising for the Germans. Their whole engineering tradition, is maybe it's too strong to say called into question, because it's there, you know, and they are pre-eminent engineers. But now, there's a paradigm shift going from petrol cars to electric vehicles, right? And the question is, will they be able to do that? So it looks like they're doing it now, but the verdict is still out.

Sandra I'm gonna ask a question that I think is on the minds of a few people. And that has to do with Agile, and going back a bit to IT projects and the way in which Agile is practised more than the way it's taught or discussed, but in the way that it's practised. There's a lot of over optimism there. It's quite costly, and that there seems to be this permanent rush to develop and deliver things before they are tested. Some of it not in keeping with what Agile really tries to achieve. So what are your thoughts on Agile?

Bent Flyvbjerg So we actually studied Agile, a little bit, just like a small study, and we want to do more. But the study that we did shows that Agile is only performing better than other products on one thing, and that is timeboxing. So they are actually better at keeping their schedule than others, which is not so surprising, because that's the whole point of agile, right, is to timebox things and then deliver within the time period that was set up. But they are not better on cost overruns and benefit shortfalls. To be clear, this is a preliminary study, it's not a big final study. But we thought this was so interesting that we would do a small preliminary study, because everybody's talking about Agile. I think personally that Agile is a good idea. Because it has this element of replication, you're not replicating the same thing. But you are replicating nevertheless, and you are using what you're learning during a short period of time during a short cycle during the next cycle. And I think that's the right kind of thinking where you have an accumulative process where you are building experience. And in what we see, and what we study it is that it's only by building this experience that you can get the positive learning curves, which are crucial. I would actually say, any project that doesn't have positive learning curves, don't do it, walk away from it, don't touch it, it's not worth it. And that's whether you are working on a product team, or whether you are, let's say a government or a big corporation that are doing projects, you just don't want to do those projects, because they are going to drain your resources and not be very successful. Make sure there are positive learning curves. Only if a project has positive learning curves, do it.

Sandra I'm very conscious of time. So maybe a last question from the audience. It takes us back to the very beginning of our conversation, and that is the Sydney Opera House. And Matt in the audience asked about projects like the Opera House that might have been a disaster in the short-term but went on to really change the environment in which they lived. They became 'lighthouse' projects. How do we account for this success or this value of something when we think about megaprojects?

Bent Flyvbjerg So this actually happened, and I'm very happy that it happened with the Sydney Opera House. Just imagine the whole story after it was completed were as bad as the story before it was completed. You know, that would be like unbearable, I mean, if you didn't have the benefits and clearly, even though the cost overrun on the Sydney Opera House was mind blowing - much larger than the Olympics. 1,400%, right? And there was a 10-year delay on the project, so very bad... Because it doesn't matter. Clearly the Sydney Opera House has made much more money for Sydney and for New South Wales and for Australia than it cost to build it. It's a treasure. And so there are projects like that, we have other examples like that in the book actually that, you know, it was a bit of a hit or miss at the outset. But it actually turned out that it became a real success. And there's no question that the benefits after justify the cost upfront. In the Sydney Opera House, I have the problem that my fellow countryperson and fellow colleague at the university I used to work at in Denmark, Aalborg University, where he was a professor, also, his career was destroyed, and he had to be a professor instead of an architect, you know, that was actually out there building buildings, and he was even a teaching assistant in Hawaii for years, you know, because he could not get a job after he was fired from the Sydney Opera House. It is not okay. That's all I'm saying. I know, this is not going to be a popular message in Australia and in Sydney. But hey, there you have it.

Sandra We can always learn to do better. I have to remind people that they can get 'How Big Things Get Done', which is your latest book, online and with Gleebooks, who is also a partner for our event. And I have to thank everyone who submitted questions, there are so so many more than we managed to get to. But if you've enjoyed this conversation about changing your mind about how you think about big projects, or previously about thinking with Danny Kahneman, you might be interested in our Unlearn Project, which is our new podcast series about changing your mind and about changing common sense. You know, why robots are coming to make your job harder, or what music is for, or why small is the new big, and you can subscribe to the online projects wherever you get your good podcasts. But Bent's provided us some fantastic insights into managing big projects, including what factors that make them work and make them successful, you know, choosing experience, choosing the right teams, thinking slow and acting fast, so much more insight in the book, including, you know, eleven heuristics for how to do better project leadership. So I really do recommend you grab a copy. Bent, it's been such a pleasure to have you. I hope the next one we do is in Sydney, and I'd be remiss not to give a very big thank you to the Sydney Business Insights team and the broader University of Sydney team, and especially Pat Norman, who is also one of your biggest fans. Ben, thank you all for spending so much time with us today and thank you for spending your evening with us.

Bent Flyvbjerg My pleasure and great questions, Sandra. Great questions from the audience, also. Thank you to Sydney Business Insights, and to you Sandra, thank you so much.

Outro You've been listening to The Future This Week from The University of Sydney Business School. Sandra Peter is the Director of Sydney Business Insights and Kai Riemer is Professor of Information Technology and Organisation. Connect with us on LinkedIn, Twitter, and WeChat. And follow, like, or leave us a rating wherever you get your podcasts. If you have any weird or wonderful topics for us to discuss, send them to sbi@sydney.edu.au