My two previous posts, looking at the reasons for deviance between management and engineer views of projects, attracted a number of comments, primarily in two camps. One camp spoke of Management-in-Denial, going into the reasons and contexts why an enterprise would behave that way, travelling into shareholder-versus-customer expectations along the way. A second camp looked orthogonally at the same shareholder-versus-customer contention, suggesting that the expectations diverged right at the start, and that we need to get better at reducing that initial divergence by concentrating on pragmatic realities. The arguments then morphed into a classic marketing-versus-engineering debate, what we used to call vapourware or markitecture or, more recently, software-by-powerpoint.
It is hard to summarise a series of comments, you can’t edit snowballs. My apologies to any and all who feel I’ve misrepresented the arguments. If you want to know what they really said, read the comments.
I get the feeling that both camps concentrate on what happens after fan and excrement are in simultaneous play, rather than in the prevention of said occurrence.
I’ve had to manage projects in some form or shape for a couple of decades now, and so far nobody in management has ever asked me to lie, even if I’ve been that “management”. Sometimes I’ve been unhappy with the techniques used to present “reality”. Sometimes I’ve been unhappy with late changes to schedules and deliveries presented to me as a fait accompli. And sometimes we’ve just got it wrong. But it hasn’t been about lying or being in denial.
How can we get better? Here are some personal musings:
1. Let the deadline influence the design
When you ask someone to do something, one of the design criteria, an important one, is time. Once you agree on time, you can negotiate the functionality and performance characteristics of the deliverable. Often you have to negotiate with a set of “incomplete contracts” but that’s life. Emotion can be taken out of the negotiating process; agile techniques can be used to learn more about detail aspects of functions and performance; there is a continuous improvement process explicit in all this, and sometimes it results in a continuous migration process, when you have yesterday, today and tomorrow systems in parallel. Time-boxing and time-placing techniques are based on this, yet for some reason get little airplay.
How come everyone got ready for things like EMU and Year 2000 and Sarbanes-Oxley and all that jazz? Because there were two projects in each case. One did the paperwork. One did the real work. The ones that did the real work knew that the dates couldn’t change, so they negotiated on the What and the How rather than the By When. Which brings me to my second point.
2. Get the signal-to-noise ratio right
More than once, I’ve been involved in projects where the governance mechanism is larger than the project team, and where the pressure placed by such mechanisms creates more paperwork than code. This does not create a divergence between management and engineer; it’s fairer to say there are two projects, one producing paper and one producing code. And sometimes there is only one team having to do both, with predictable results.
As long as we have an Armaggedon approaching for the battle between professions, and as long as organisations haven’t completed their painful trip from hierarchy to network, we’re going to have intermediaries. And there is a price to pay for having them. The only solutions I can see are twofold: One, try and make the doer-to-checker ratio an integral part of the reporting pack, so that the cost becomes visible. Two, use the remaining checkers wisely, they can help in many ways. Which brings me to my third point.
3. Pass ownership to the real customer as soon as possible
We use terms like fast iteration, early testing, agile development, fail-fast, whatever. Hitherto I’d been worried about the tension between such approaches and the classical waterfall or cascade approaches. But the more I think about it, the more I feel that the approach-conflict is but a symptom. The root cause is a blame culture. Which sadly pervades too many organisations. So what we need to do is to find ways of reducing the impact of this blame culture. As far as I can make out, the best (perhaps the only) solution is to place something that works in the hands of the real customer, that can be used to create or derive value. Once you do that, all the dynamics change. Even if the value is marginal to begin with. Which brings me to my final point.
4. Get legacy skin into the game as soon as possible
Don’t talk parallel run. Don’t talk big bang switchover. Carry a big stick, move low-volatility information from Old to New as soon as humanly possible. Expose what you’ve done to the customer, and soon the old will atrophy and the new will thrive. [Note: There is an exception to this, an important one. Sometimes you get involved in building systems that are expected to do precisely what the ones they are meant to replace do in the first place. To the letter. No point getting legacy skin into this game, the customer doesn’t want the system. Get out quick.]
I’m looking for JP’s email address for my CEO Pip Coburn – would appreciate any help –
Thanks,
David
David, you should have an email from me. Rgds
JP, I am surprised that you passed up the perfect opportunity to analogize your favorite Cluetrain slogan! In this case it is not MARKETS that are conversations, but DEVELOPMENT PROJECTS! I believe strongly that the ABSENCE OF CONVERSATION can account for unrealistic expectations, inadequate requirements analyses, unsatisfactory testing, and, of course, completely fictitious time-lines! If you are not familiar with the concept of participatory design, the Wikipedia entry is a great place to start:
http://en.wikipedia.org/wiki/Participatory_design
This replaces the blinders of a pipeline operation that channels information from client to requirements analysis to design to engineering to testing to client delivery. In its most idealized form the entire process becomes an ongoing conversation among all of the participants in the production process along with representatives of the intended beneficiaries. (I have even seen an example that includes maintenance and repair of the not-completed product as part of the conversation.) There are those who argue that this takes more time than the conventional pipeline (or waterfall) model; but such critics usually do not factor in the time invested in compensating for the failed results and salvaging them into something the client can actually use. The REAL problem is that participatory design shifts the paradigm from the impersonal production-line models we all know and love (SIC) to social models of communication and action through conversation; so the usual reaction in most development shops comes down to, “We can’t do that.” I wish that better social software could enable a paradigm shift, but you now know me well enough to recognize that I am not a particularly rosy optimist!
A big one this JP! There are many reasons a project fail some of which have been mentioned:-
1) Misunderstood or vague requirements
2) Over optimistic planning
3) Too much emphasis on project reporting and not enough on project management.
4)Machiavellian vendors and project managers
5) Inability to fail quickly and have another go
1) can be overcome by becomming more ‘agile’ and less waterfall in approach – getting the real end users involved earlier in the process – regualr testing and review etc. Pure waterfall has been discredited for a long time in the IT community and it amazes me that it is still proposed as an option… Of course with waterfall the testers can blame the developers who can blame the designers who can blame the original specifications etc… so thats OK then…
2) Is more interesting. I have seen this many times with relatively inexperienced tech leads who are very very good coders. They have always coded their way out of problems and hence always shipped good product on time. Unfortunately this doesn’t scale when they run a team who are not as ‘talented’ as them. Good and experience managers will nuture this talent rather than allow it to get burnt out and blamed for failure.
3) Big rudder steering a small boat syndrome. You dont get much done and the project fails out of frustration. ‘Why do have have 25 people on the project and nothing is being delivered?’. Well maybe because you have 15 strong project reporting team telling you where you are and 7 architects, strategists etc telling you where they should be going. Mean while the 3 programers are trying to justify the 50 dollars purchase of a set of oars. The answere to a failing project is not simply to add more project reporting as frequently happens. Better project management maybe but more reporting is just telling you more accurately what you already know (athough it does highligh who to blame…)
4) is real! How many times have we seeen big government contracts awarded to big contractors who have a history of failure? They get paid for delivering a requirements doc. They get paid for delivering a technical architecture. They get paid for delivering hardware. When they hand off to the software development team the blame game starts. Sometimes it gets as far as the testing team .. BUT in any case so much money has been sunk into the project the managements and the sponsors (who haven’t seen a functioning system yet) don’t want to admit they have been lead up the garden path for fear of being found out to be lacking themselves. SO we go through another round of expensive change requests and new features to justify the extra cost. More robust management anyone…and how about less throwing good money after bad and throwing good money at good development teams and projects?
5) It amazes me that hard nosed investment bankers who will cut a loss making position in an instant are unable apply the same logic to their IT investments. Instead they buy into the myth that the next injection of capital will be the last and that the projects will inevitably succeed simply because they want them too. This to is rapidly followed up by the blame game… IF you have to fail at least try to fail cheaply!
at the end of the day, it’s still about people managing people, and people managing other things
…the best laid plans. And there I was planning to close with an It’s all about Trust. And relationships. Covenant not contract. Relationship before transaction. Conversation before execution. And so on.
I’ll probably do a closing post anyway, but you guys have said it all for me :-)
JP,
It is always interesting when I find many unrelated people talking about the same topic.
Here are two article from my part of the software world.
http://www.embedded.com/showArticle.jhtml;jsessionid=B5O2P13K0LXNOQSNDLPSKH0CJUNN2JVN?articleID=193302974
http://www.developer.com/java/ent/article.php/10933_3635191_1
I also want to point out the very real human cost of a project failure. I have seen engineers work such insane hours that they never see their 2 year old when he is awake. That is morally bankrupt management.
There is not top down solution. It has to happen at all levels. I am a personal follower and big fan of the Personal Software Process from CM and Watts Humphrey. That addresses the problem at the engineer level.
The tools are known, but seldom used.
“It’s all about Trust. And relationships. Covenant not contract. Relationship before transaction. Conversation before execution.”
in a utopian world this is absolutely correct of course! This is essentially predicated on all partners having the same aims and goals. In the big bad world this is not always the case. The software vendor wants to make as much money as possible and hence wants to lock the purchaser into an unfavourable contract. Even with internal developments – politics and personal goals can cloud the picture…
I remeber one large project in which stake holders and project management agreed was ‘on track’ to deliver. When it became obvious that this was not the case, I took over and reprojected at double the cost and2.5 times the orginal time. One very revealing comment I got from several very senior executives was that they always new it was going to over run but they had been forced to reduce the estimates to ‘get project approval’. Interestingly the original estimates were very similar to my reprojection. In this case their personal goals were very different from the ‘project approvers’. The ‘approvers’ reluctantly agreed to continue the project as a lot of money had already been spent and the project delivered as per the new projection. At the end of the day the result was that the machievellian manovering had paid off.
Of course project should not be run this way and it is encumbant on management try not to let it happen.
But it is not that easy in real life. Too often we end up ‘trusting’ the untrustworthy, entering into covenants with those who will break them. That’s why relationships are as important in business as they are in every other aspect of life. A good relationship will have shared goals and will last far beyond an individual project.